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ABSTRACT 

,_ T ._. The 18-month Skills Today for Advancement Tomorrow 

WAT) program, a partnership among St. Louis Community College, the 
St. Louis Public Schools' Adult Basic Education Program, and Blue 
Cross and Blue Shield of Missouri, had the following objectives: (1) 
provide counseling and training for 370 current Blue Cross and Blue 
Shield workers; (2) develop a training referral system for 60 
unemployed disadvantaged adults; (3) develop model assessment and 
evaluation systems for health care benefits workers; (4) develop and 
validate competency-based basic skills curriculums; and (5) develop 
a r ci m S} ement innoVative methods for involving workers in all aspects 
of STAT s program development. During the program, 515 participants 
were served with classes for the Blue Cross/Blue Shield workers; 60 
clients were served through the partnership between the Adult Basic 
Education program and the STAT program; model assessment tools and 
model curricula were developed; and worker involvement was encouraged 
through one-to-one counseling, feedback questionnaires and focus 
group sessions. Participants from the insurance company improved 
their performance ratings and customer satisfaction ratings. (Project 
documents, including questionnaires, a sample curriculum and a list 
of dissemination activities, are included in the report.) (KC) 
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Skills Today for Advancement tomorrow (STAT) Project 
PR Award V198A30199 
Final Performance Report 
March 20, 1995 

The Skills Today for Advancement Tomorrow program 
(hereinafter referred to as STAT) was funded from March 1, 1993 through 
December 31, 1994. (The original grant that was funded for 18 months 
was allowed a no-cost extension affording four additional months for 
program operation.) Partners included the Blue Cross and Blue Shield of 
Missouri (BCBSMo), the St. Louis Public School's Adult Basic Education 
Program (ABE), and the grantee, St. Louis Community College. 

The STAT goal was to focus on the workplace literacy training 
needs of the BCBSMo health care benefits industry. The five objectives 
for the project were to: 

(1) Provide counseling and training for 370 current Blue Cross and Blue 
Shield of Missouri workers, including literacy skills such as reasoning, 
concentration, memory, problem solving, reading, writing, speaking, 
listening, and computation skills (300 workers) and pre-technical 
"imaging" skills (70 workers); 

(2) Develop a recruitment and training referral system for 60 
unemployed, disadvantaged adults to gain those skills necessary to 
successfully compete as applicants for employment at BCBSMo or 
similar work environments; 

(3) Develop model assessment and evaluation systems which will create 
new qualitative and quantitative measurement tools for health care 
benefits industry workers; 
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(4) Develop, validate, refine, reproduce, and disseminate systematic basic 
skills curricula that are competency based and transferable to other 
similar industries; 

(5) Develop and implement innovative methods for involving workers in 
all aspects of STAT's program development. 

During the STAT grant numerous changes occurred in the Blue 
Cross and Blue Shield of Missouri organization. The company moved its 
headquarters to a new, downtown location. Two downtown St. Louis 
sites housed the BCBSMo staff. Considerable down-sizing and 
restructuring occurred within all departments, and the company changed 
its status from a not-for-profit organization to a publicly licensened and 
incorporated company, now called Alliance Blue Cross and Blue Shield. 
Since STAT was operational at the time of the company's transitional 
period, we were positioned to provide much-needed services to this 
workforce. The partnership provided basic skill enhancement that 
assisted workers keep jobs, gain skills needed to compete for new jobs, or 
prepare to find jobs elsewhere. The project's objectives, while doable, 
needed to conform to the perceived needs of the company and were 
adjusted to the culture of the company over the project period, as any 
viable workplace literacy program must do in order to be successful. 
Hence, some of the planned details and priorities had to be augmented 
and refined to apply to the "real world" of this workplace. 

Two educational partners working with one business partner also 
provided many learning experiences. Merging three separate 
organizational structures into one "holistic" team was at times awkward 
and difficult. Conflicting needs, such as the Adult Basic Education's rigid 
enrollment requirements of attendance, participation numbers, testing 
procedures, and reporting procedures were not known in advance of the 
project's implementation. Though efforts were made to adjust the 
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program to "fit" these real constraints, the scope of the Adult Basic 
Education program's participation was narrowed from the projected plan. 
However, the STAT program provided ABE with a real opportunity to gain 
new insights into workplace literacy training, and ABE shared many 
responsibilities for program implementation including staff selection, 
program planning (especially in regard to how we could serve the external 
participants), and project evaluation. At times all three organizational 
needs could not be met, and in the STAT program, the BCBSMo needs 
took priority over the educational organizations' needs. 

The process for integrating St. Louis Community College staff with 
BCBSMo staff was dynamic and, though at first quite slow, gained 
momentum throughout the project. St. Louis Community College 
personnel worked alongside BCBSMo personnel. Daily planning and 
implementation of each activity blended roles for both Coordinators (one 
Project Coordinator a*id one BCBSMo Coordinator). The ultimate 
evidence of the full cooperation between the BCBSMo and St. Louis 
Community College staffs was in the evaluation data collection period 
toward the end of the project. It was due to this organizational 
"harmonizing" that many outcomes were realized in the STAT project. 

The project evaluation was conducted by Dr. Sharon Slane. Her 
outstanding contributions provided guidance during the formative 
evaluation period and through the completion of the summative 
evaluation. Dr. Slane participated in partner meetings, giving direction 
toward the stated objectives. She observed every facet of the program in 
operation, led focus group feedback sessions with participants, 
supervisors and staff, composed the data collection system instruments, 
provided a timely mid -project report (the formative evaluation report), and 
the final evaluation report. In brief, Dr. Slane fulfilled her role as the 
"objective" evaluator. Her efforts heightened the project's efforts 



throughout the funding period, and with the final report, documentation 
is available which will doubtless benefit St. Louis Community College's 
future workplace literacy efforts as will the work of other workplace 
literacy programs, when applied. 

Since a comprehensive final evaluation report is provided, specific 
details, including the data collection system, instruments, and project 
outcomes, are available in that document. This report will detail some of 
the learning gained through the partnership, the staff development 
activities (which provided excellent professional growth for the 
instructional team), and dissemination efforts that were provided during 
the grant and have been continued throughout these months beyond the 
grant period. 

Per the instructions provided by the U. S. Department of 
Education, the following segments in this report respond to the "reporting 
requirements" and are identified as Roman Numerals I through VI. 

I. Actual accomplishments related to the specified objectives are: 

Objective 1: 515 participants were served during the grant (370 were 
specified). Courses were developed to respond to the workers' needs 
including brush-up courses in reading and math, effective 
communication skills, memory and concentration skills development, 
and study skills. Class formats were highly innovative with a variety of 
formats including short courses being provided during lunch times, brief 
seminars, "take home" packets, and workshops. BCBSMo purchased 
computers, and the STAT program assisted in the implementation of a 
computerized learning and resource center. Workers attended classes 
voluntarily at two work sites, both within the BCBSMo environments. 

Two significant project outcomes are the institutionalization of the 
STAT program at Alliance BCBS, and the release time that is now 
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provided to workers who participate in STAT courses (on-going classes 
are now provided by Alliance BCBS with purchased services from St. 
Louis Community College when needed). 

The "imaging" classes which were originally planned were not 
provided through the grant; however, BCBSMo provided the training that 
was required to get the workforce up-to -speed on the new "imaging" 
system. 

Objective 2: 60 clients were served through the partnership between the 
St. Louis Public School's Adult Basic Education program and the STAT 
program. Since BCBSMo was downsizing during the grant period, a 
hiring freeze was in operation at this time, and no "external" applicants 
were considered or employed by the company. A great deal of planning 
was given to the issue of upgrading unemployed worker skills with "real" 
workplace competencies. The course provided to the 60 external 
participants included a modified workplace literacy curriculum on 
communication skills, problem solving skills, and technical reading. 
Immediate feedback from participants was positive. Unfortunately, no 
process to gauge actual transfer of this training to workplace skills was 
developed or utilized with this population. 

Objective 3: Specific model assessment tools are included in the final 
report. Significant outcomes are provided in the final evaluation report. 
Of particular note are: 

• A curriculum analysis tool that was developed and distributed to 
teachers external to the STAT project. Copies of the curricula were 
studied and comments duly noted. Relevancy, quality of organization, 
appropriateness of reading levels for the materials, and transferability 
were criteria which the teachers utilized for evaluating curricula. (See 
Final Evaluation Report for forms and responses.) 

5 
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• Questionnaires were provided to participants at various intervals 
during the project and to the supervisors at the project's end! The 
BCBSMo policy regarding confidentiality for workers who attended 
STAT classes prevented a full involvement by supervisors. However, 
one project outcome is that Alliance BCBS now incorporates both 
employee and supervisory involvement in the educational program. 

• Program participants' Selection System* test scores, retention and 
promotion rates, claims and customer service job performance ratings, 
productivity and customer service ratings were analyzed for significant 
differences as a result of training. These results provide positive 
project outcomes regarding STAT's impact on the individual 
participants and the company. (* The Selection System includes 
testing on the Selection Exercises which are tied to the unique 
BCBSMo workplace competencies for reading, math, writing, problem 
solving, and concentration/memory skills. The Selection Exercises 
(assessment tests) are given by the Personnel office to workers. 
Results from the Selection Exercises provide one criteria for the 
workers' ability to compete for other available jobs. Results were 
shared with STAT staff who then utilized results to counsel 
participants.) 

Objective 4: Model curricula were developed, refined, reproduced, and 
disseminated to various organizations. These materials are available on a 
printing cost-recovery basis to any similar industry or educational 
organization that requests them. (A sample of the customized curricula 
may be found in Attachment 1.) 

Objective 5: Worker involvement was encouraged through: 

• One-to-one counseling that was provided to workers by project staff. 
This included identifying and reporting skill deficits as measured on 
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the company's Selection System instruments, developing individual 
educational plans, and providing instruction through a variety of 
formats. 

• Feedback questionnaires and focus group sessions that were utilized 
to gain more information from participants. The information was then 
utilized to improve instruction and afford courses which were relevant 
and timely to the workers. 

The staff development activities strengthened the professional skills 
of all STAT staff while building instructional team skills. Special 
activities were: 

• Partner meetings, including a one-day planning and implementation 
"retreat" 

• A two-day orientation for project staff, and the ABE teacher 
certification workshops; 

• Weekly staff meetings conducted by the Project Coordinator; 

• Teleconferences (on Learning Disabilities and Adult Numeracy) 

• Interaction with other programs and staffs e.g. Tulsa Technical 
College, University of Missouri- St. Louis, East Mississippi Community 
College, Texas Instruments Workplace Literacy project, Regional 
Commerce and Growth Assn. of St. Louis, AAIM Management, local 
labor organizations, United Way Task Force on Literacy, Gateway to 
Literacy, and ASTD- Train America's Workforce committee and local 
meetings, National Governor's Conference-Education Policy 
Forum; 

• Visits made to the STAT program including L'.z Miller, Program Officer 
from the U. S. Department of Education, and Susan Rosenblum who 
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conducted a field test at the STAT project for Cosmos' National 
Workplace Literacy Program evaluation. 

• Staff attendance at a number of trainings provided by external 
organizations, including the Coordinator attending Pelavin 
Associates - U. S. Department of Education training, teachers 
attending the ICANS training held by North Kansas City ABE and the 
Northwest State's ABE programs, the Director and Coordinator 
attending the Institute for Workplace Literacy conducted by the 
Colorado Consortia, the Director's participation in the Missouri 
Literacy Technology Summit, the Director and Coordinator's 
paiticipation at the Literacy and Work Roundtable (Nashville, TN - 
AAACE pre-conference). 

II. Once staff were employed and initial training conducted, the schedule 
of accomplishments followed the specified plan. Classes were on-going 
throughout the project period. Our evaluator assisted with establishing 
evaluation data collection and monitoring systems. 

III. The Final Evaluation Report details characteristics of project 
participants and participant outcomes. Briefly, 515 worker participants 
and 60 external participants (ABE students) were served during the grant 
period. The mean age of worker participants was 35 years old with 
race/ethnicity being 60% white and 40% black. Approximate years of 
service with BCBSMo was 8-10 years, 10% average, 1 1-15 years, 50% 
average, and 16+ years, 40%. A higher percentage of female participants 
were served which parallel the BCBSMo demographics: 71% female to 
29% male. 

IV. Dissemination activities were many and varied. They include 
presentations at conferences, to businesses or organizations which were 
made by the Director or by the Director and Coordinator, convened 
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meetings with businesses, responses to individual requests for 
information, newspaper articles, and participation as a panel member for 
the National Center on Adult Literacy's national teleconference on 
workplace literacy. (See Attachment 2 for a listing of dissemination 
activities). 

V. Evaluation activities were central to the success of the project. 
Beginning early in the project, the external evaluator provided guidance 
for the development of the evaluation system, collection instruments, and 
a timeline. The formative evaluation was conducted in the early spring 
1994, one year into the project. With these results we were able to adjust 
our training to effect needed improvements. All partners were kept "on 
track" regarding the project's planned objectives; all had input into how 
we would evaluate outcomes. The summative evaluation activities 
intensified during the last quarter of the project. Staff were into "file 
diving" (a term coined by the Project Coordinator to express the intense 
work which had to be done) to gather individual data on participants 
related to job performance. The focus group sessions that were 
conducted during the formative and summative periods provided input 
from participants after they had had an opportunity to put newly 
acquired skills to work and reflect for a time on what the training had 
meant to them. We found focus group sessions a most valuable way to 
measure longer-term outcomes of our training. Supervisors were candid 
during the focus group sessions also. They all wished that they had been 
involved earlier in the STAT program; still, their comments were very 
positive about the STAT program. (See Final Evaluation Report for 
company impact.) 

VI. The original application had two part-time workers identified: one for 
counseling and a second for curriculum coordinator/master teacher- 
trainer. Per our request for a project revision on May 25, 1993, approval 



was given to substitute one full-time position, Project Specialist. 
Resumes for all staff and the external evaluator were provided. 

The original federal budget for the STAT project was approved for 
$285,125. The actual expenditures amounted to $245,236.31 in federal 
funds and a match amount of $116,092,87 (32%) supplied by the 
grantee. 

St. Louis Community College wishes to gratefully acknowledge the 
opportunity provided by the U. S. Department of Education's National 
Workplace Literacy Program to develop the STAT program. Although 
much has been learned through this and other national workplace 
literacy projects, the whole field of workplace literacy is barely beyond 
fledgling status. It is important that successful programs show a return 
on investment in order to gain true "buy-in" by business, as ultimately 
business must pay for up-grading worker skills. And the ability for 
programs to track and evaluate the transfer of learning, both to workers' 
jobs and beyond to improved personal and family life, is the issue which 
drives us, as providers, to continue our work in the field. Beyond these 
pioneering efforts, more model programs need to be showcased. As 
educators,. the field provides us with continuous learning opportunities. 
For assisting the workforce of tomorrow, these efforts cannot be 
underestimated as to importance or impact on the future of our economic 
and social health. 
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Attachment 1: Sample STAT Curriculum 
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Sample STAT Technical Reading Skills Curriculum 
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Exercise 16 Skills: Drawing accurate conclusions; Using logic to interpret written 
material. 



Drawing Conclusions - Exercise 16 



Directions: Respond "A" for correct conclusions, "B" for incorrect conclusions, or M C 
for inconclusive evidence based on the facts given. 



Facts: 



Doctor A charges less for an operation that Doctor B. Doctor B 
charges less than Doctor C. 



Conclusions: 1. _ 

2. 



Doctor C charges more than Doctor A. 

Doctor C charges more than either Doctor A or B. 



Facts: 



Hospitals W and X charge with same amount for a given procedure. 
Hospital Y charges twice as much as Hospital Z. Hospital Z charges 
less than Hospital W. 



Conclusions: 



3. 
4. 
5. 



Hospital Z charges less than Hospital X. 

Hospital Y charges twice as much as Hospital W. 

Hospitals W and X charge $500 for the procedure in 
question. 
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Sample STAT Punctuation & Grammar Skills Curriculum 
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WRITTEN EXPRESSION EXERCISE 

Proofread the following material. Correct any errors in spelling, 
punctuation, grammar, and word usage. 

The mission of Blue Cross and Blue Sheild of Missouri is to 
manage health care benifit programs, which will ensure affordable 
excess to quality health care. 

Understanding that our business existence and continued 
success are dependant upon how good we meet our responsibilities 
to our constituencies we will continually strive to acheive a high 
level of client's satisfaction from all group served. 

Our primary responsibility are to our customers on who we 
place an overriding importance because them, and them alone, 
allows us to exist. 

We will always deal with our customers according too the 
highest standards of Fairness and integrity We will treat our 
customers with respect and dignity, and offer you only quality 
products at fair prices. 

We will also strive to ensure that our members have 
affordible access to the best that medical technology and care has 
to offer: and act to assure that they recieve consistently high quality 
service. 

We will always respect the dignity and privacy of our members 
by assuring confidentiality in regards to there medical an financial 
records. 

We will treat our members with promptness courtesy, equality, 
and fairness in all our dealings with them. 



Punctuation 


Grammar 


Usage 


Spelling 
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Sample STAT Perception Concentration Curriculum 
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PERCEPTION CONCENTRATION 



I. Accessing Information by Computer Code 



This set of exercises requires you to pay very close attention to detail in the 
use of numbers and letters. These exercises will help you practice the 
following skills: 

Reading and following instructions 
Attending to detail 
Matching information 
Transferring information 
Increasing speed 

Note: The Answer Key for all exercises is in the back of the instructional 
packet. 
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These are the directions for the first exercise. We will lead you through 
these directions step-by-step so that you can improve your skills in reading 
and following written instructions. 



Exercise 1 



Time: 5 minutes 



In this test you are to use the code chart that 
appears below. The chart shows the codes 
used for the claim amount submitted by the 
customers listed in the claim column. To find 
the comet code for each subscriber, you 
shoulc(nfsj)iogkat the name in the claim 
column ancflherafind the identical name in 
the subscriDeTcolumn.(Next,)ook at the 



amount of that subscriber'sclajm submitted 
in the claim amount column^No^/find that 
amount in the code chart. Write the 
corresponding code letter in the space 
provided after the subscriber's name in the 
claims column. See the example below and 
study it carefully. 



Claim Codes: 




$499.99 or less 


L 


$500-$999.99 


M 


$1,000-$1 ,499.99 . . 


A 


$1,500-1,999.99 


N 


$2,000 or more 


O 


Not listed 


R 



Subscriber 
Example Mabley, J. D. 



Claim Amount 

$1,534 



Claim 

Mabley, J.D. 



N 



1) LOOK FOR THE KEY WORDS AND PHRASES IN THE DIRECTIONS. Key words 
and phrases contain essential information. In the above directions, the key words and 
phrases have been boldfaced. 

2) LOOK FOR ANY SIGNAL WORDS. Signal words tell you the order of the steps 
needed to complete the exercise. In the above directions, the signal words have been 
circled. 

3) Finally, ALWAYS LOOK AT THE EXAMPLE PROVIDED. The example lets you 
check your understanding of the directions. 
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Exercise 6 Skills: Reading and following instructions, attending to detail, matching 
numbers, proofreading, increasing speed. 



Exercise 6 



TIME: 5 minutes for Exercises 6-8. Check answers at the end of Exercise 8 or the 
end of allotted time. 

For each of the 15 items below there are three numbers that you should compare 
carefully. Decide if all three are alike, if only two are alike, or if none are alike. Crcle 
the proper number in the answer column to indicate your decision as shown for item 0. 

Exercise 6 

ANSWERS 



0. 


12579 


12579 


12579 


0. 


3 


2 


0 


1. 


1613594 


1615394 


1613549 


1. 


3 


2 


0 


2. 


87934 


87934 


87934 


2. 


3 


2 


0 


3. 


140031674 


140031764 


140031674 


3. 


3 


2 


0 


4. 


932467 


932467 


932647 


4. 


3 


2 


0 


5. 


0401793.2 


040179.32 


. 0401793.2 


5. 


3 


2 


0 


6. 


89005237 


8900523V 


89005237 


6. 


3 


2 


0 


7. 


1.397.361.2 


1.597.362.1 


1.957.362.2 


7. 


3 


2 


0 


8. 


54921.321 


54921.321 


54921.211 


8. 


3 


2 


0 


9. 


0.2.37893 


0.2.38793 


0.2.38793 


9. 


3 


2 


0 


10. 


1378934 


1387934 


1397834 


10. 


3 


2 


0 


11. 


247.398.21 


247.389.21 


247.398.21 


11. 


3 


2 


0 


12. 


39546.234 


39546.234 


39546.234 


12. 


3 


2 


0 


13. 


79812 


79182 


79812 


13. 


3 


2 


0 


14. 


394.250.32 


394.520.32 


394.230.23 


14. 


3 


2 


0 


15. 


453042169 


453402169 


453402169 


15. 


3 


2 


0 
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Exercise 18 Skills: Attending to detail, locating information, transferring information, 
drawing conclusions. 

Exercise 18 
Performance/Cash '91 

SCHEDULE OF ESTIMATED PAYOUTS 



If your 
individual 
performance 
rating is: 



And you are: 



And BCBSMo's 
weighted Service 
Performance is at least 
90% and Financial 
Performance is a net 
pre-tax income of: 



Then your 

Performance/Cash '91 
bonus payment is 
estimated to be: 



2% of your base salary 
3% of your base salary 
5% of your base salary 



Consistently 
Surpasses 
Standards 
(CSS) 



Exempt 



$25 million 
$30 million 
$35 million 



Non-Exempt 



$25 million 
$30 million 
$35 million 



Consistently 
Meets And At 
Times Exceeds 
Standards 

(CMS) 



Exempt 



$25 million 
$30 million 
$35 million 



4% of your base salary 
6% of your base salary 
9.5% of your base salary 



1 .5% of your base salary 
2% of your base salary 
3.5% of your base salary 



Non-Exempt 



$25 million 
$30 million 
$35 million 



Meets 
Minimum 
Standards 
(MMS) 



Exempt 



$25 million 
$30 million 
$35 million 



3% of your base salary 
4% of your base salary 
6.5% of your base salary 



.75% of your base salary 
1% of your base salary 
1 .5% of your base salary 



Non-Exempt 



$25 million 
$30 million 
$35 million 



1 .5% of your base salary 
2% of your base salary 
3% of your base salary 



CSS, CMS, 
or MMS 



Exempt or Non- 
Exempt 



Less than $25 million 



$0 



CSS, CMS, 
or MMS 



Exempt or Non- 
Exempt 



More than $35 million 



Same as with a net pre- 
tax income of $35 million 



Please note that these are estimated payouts to be made in the first quarter of 1992 after the 1991 
year-end results are known and reviewed by the BCBSMo Board of Trustees. 

"Base salary" includes a lump-sum merit increase, if any, earned in 1991 and to be added to your salary 
in 1992. Bonus payments will be subject to all federal, state and local taxes and FICA (Social 
Security) deductions. 

Source: Blue Cross Blue Shield of Missouri 
q 39 
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Exercise 18 (cont'd) 



1 . An exempt employee whose last performance rating was MMS can expect a 
Performance/Cash '91 bonus payment of how much if BCBS's 1991 financial 
performance is a net pre-tax income of $35 million? 



2. Which employee can expect the highest Performance/Cash '"91 bonus payment 
offered by BCBS and under what circumstances? 



3. What is the dollar amount of the lowest possible net pre-tax income of the Financial 
Performance listed? 



What is the estimated BCBS bonus payment in the event that the net pre-tax 
income is less than this above-stated dollar amount? 



4. Sally K. is a long-term non-exempt BCBS employee who consistently receives 
individual performance ratings of CMS. Her base salary is $24,500 in addition to a 
$2,000 merit increase she will receive in 1991. In December 1991, Sally K. inquires 
how much her bonus payment will be, if her merit increase will be included in her 
salary amount, and if her unspecified bonus will be subject to taxation. 



40 
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Sample STAT Business Letter Writing Self-Study Packet 
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Business Letter Writing Self-Study Packet 



The attached packet allows you to practice your business letter writing 
skills. The packet includes the following: 1) Errors Table; 2) Tips on 
Writing a Business Letter, and 3) Practice letters A - E. 

Errors Table 

The Errors Table on the next page lists the criteria for scoring the Writing 
Skills Exercise given by BCBSMo Employment and Placement. STAT will 
use the same criteria and scoring guide in assessing your practice letters. 
Two main factors are considered in the scoring process: language 
mechanics and expressiveness. The bold italicized text in the Errors Table 
indicate re-occurring problem areas that we see. Be especially careful to 
make your sentences and your entire letter very clear as well as to include 
all information given in the facts. 

Tips on Writing a Business Letter 

Read through these tips before beginning your practice letters. They 
provide definitions and explanations of the various parts of a business letter 
as well as good and bad examples of effective business letter writing. 

Practice Letters A -E 

Permit yourself 20 minutes to read each letter and write your response. 
This will allow you to practice thinking and responding quickly yet 
accurately. As you finish each response, return your practice letter to 
a STAT instructor. He/she will score your letter and provide feedback. Use 
the feedback given in each returned letter to respond to the next letter. DO 
NOT DO ALL THE PRACTICE LETTERS AT ONE TIME. Practice will help 
you build your writing skills. 
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Errors Table 



1 Factor 1: Mechanics 


Examples: 


1 . Spelling/Word Usage (deduct 5 points 
for each occurrence) 

2. Punctuation/Grammar (deduct 5 points 
for each occurrence) 

h — = — 


1 . Misspelled word; misused word (e.g., 
their for there, accept for except, 

i iron a rH loco at/* \ 

2. Incorrect or missing punctuation marks; 
excessive use of commas; incorrect 
subject-verb agreement or other syntax 
errors. 1 








Factor 2: Expressiveness 


Examples: 


1. Clarity: (deduct 40 points If 
Unacceptable; or zero points If 
Acceptable) 

2. Tone& Style: (deduct 20 points if 
Unacceptable; or zero points if Acceptable) 

3. Responsiveness: (deduct 40 
points If Unacceptable; or zero 
points if Acceptable) 


1 . Did the aDDlicant make the resconse 
clear and easy to follow? Is the letter 
logical? (Look for unclear 
sentences or unclear expression of 
Ideas.) 

2. Did the applicant demonstrate sincere 
concern for the customer's problem? (Look 
for examples of haughtiness, or a lack of 
tact.) Is the letter polite and business-like? 
(Look for examples of sarcasm, arrogance, 
belittlement, or rudeness.) 

3. Are all the customer's questions 
or problems answered In the letter? 
(Look for missing Information 
regarding customer's questions.) Is 
the customer advised as to how the 
problem is or will be resolved? (Look for 
missing advisory statements or missing 
summary describing the status of the 
customer's problem.) 
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TIPS ON WRITING A BUSINESS LETTER 



Greeting 

• Address the person by name (use Ms. if a woman is not married or if you are 
uncertain of her marital status). 

• Use a colon after the greeting. 

Opening 

• State the purpose or reason for the letter. 

• If it seems appropriate, thank the person for his or her letter/ inquiry. 

• If you received a complaint you might simply say that you received the letter 
dated — and remind the reader of the content of his or her letter. 

Body 

• Be sure you provide adequate information to answer the inquiry. 

• Give a reasonably full explanation, providing reasons for certain costs and 
procedures and showing how you arrived at percents and amounts. 

• Follow a clear, logical order in presenting facts; be careful not to omit any facts 
and give information step by step. 

• Be careful not to provide more information than requested; do not 
overwhelm the reader with extra or irrelevant material. 

• Explain any action the reader needs to take. 

Conclusion 

• Reinforce the purpose of the letter and/ or thank the reader. 

• Offer further assistance if the reader needs it 

Tone 

• Always be courteous, cordial, and professional. 

• Try to convey concern and willingness to help. 

• Emphasize the positive and minimize the negative. 

• Defuse any hostility as much as possible. 

Style 

• Be clear and concise as well as accurate and informative. 

• Use correct grammar (subject-verb agreement; appropriate pronoun, adjective, 
and adverb forms) with correct spelling, punctuation, and word choice. 

• Do not use contractions or slang. 
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LETTER A 



January 18, 1994 

Blue Cross Blue Shield of Missouri 

1831 Chestnut 

St Louis, MO 63103 

Dear Sin 

I have not been in this area for a very long time, and I have just finished my first six 
months of employment at Company ABC. The person in charge of benefits told me a 
month ago that my dependents and I were eligible at that time for health insurance 
coverage. 

I had a surgical procedure done in a doctor's office two weeks ago, and it cost me 
$1,375.00. I submitted a claim to you right away. I just got a letter back saying that I 
am covered for only $500.00. I know you pay only 70% of all charges, but do you also 
limit all procedures to only $500.00? I paid the doctor, but now I only got part of it 
back from you. My company says I have insurance, but all you paid me was a lousy 
$210.00! 

Sincerely, 

LM. Mad 



FACTS ABOUT THIS INQUIRY 

1. The office surgery performed on Mr. Mad related to a preexisting medical condition. If Die 
patient is insured for fewer than 12 months by us and had no previous insurance coverage - 
as is the case with Mr. Mad - a limit of $500.00 reimbursement per person is placed on 
procedures relating to preexisting conditions. 

2. He is correct about the 70% coverage, but he is not aware that lie has a deductible of $200.00. 
This deductible must be satisfied only once per person per calendar year. 

3. From the $500.00 maximum eligibility for preexisting conditions, his $200.00 deductible 
must therefore be subtracted. Tliis leaves $300.00. We pay 70 % of the $300.00, or $210.00. 
The difference between his $1,375.00 surgery bill and the $210.00 that we pay is his 
responsibility. 

4. Tlte full explanation appears on p. 15 of his benefits booklet. 
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Attachment 2: Dissemination Activities 
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STAT Dissemination Activities 

The following is a listing of dissemination activities that were provided 
through the STAT National Workplace Literacy program: 

Presentations to the following organizations (chronological order): 

University of Missouri Extension's Annual Conference 

UAW Local #2250 Community Services Committee 

MO's 1 1th Annual AFL/CIO Community Services Conference 

AAACE November 1993 Conference 

LIFT-MO's "Collaborations for Success Conference" 

NCEA Region IV Community Education Conference 

National Workplace Literacy Program, Business Partners Breakfast 

United Way Labor Counselor Class 

Missouri Vocational Association Annual Conference 

Missouri Community College Association's Annual Conference 

Webster University Student Literacy Corps Class 

Illinois Partnership for Workforce Education Conference 

League for Innovation in the Community College, Workforce 2000 

NCAL's What Works? Literacy Training in the Workplace Teleconference 

Technical Assistance and/or Materials (alpha order): 

Alumax of South Carolina 

East Mississippi Community College 

Edwardsville Alternative School 

Monroe Community College 

Ranken Technical School 

Red Cross of St. Louis 

St. Mary's Hospital 

Train America's Workforce Committee of ASTD's Local Chapter 
Newpaper Feature Articles: 
Riverfront Times, March 15, 1995 



9 

ERIC 



31 



Dissemination of Report: 

Institute for the Study of Adult Literacy 
Dr. Larry Mikulecky, Indiana University 
National Center for Adult Literacy 
ERIC 

LIFT-MO, Missouri State Resource Center 

AAIM Management of St. Louis 

Regional Commerce and Growth Association 

United Way of Greater St. Louis 

Literacy Service Center of St. Louis 
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EXECUTIVE SUMMARY 



In the spring of 1993, the St. Louis Community College was awarded a National 
Workplace Literacy grant from the U.S. Department of Education to implement the STAT 
Program (Skills Today for Advancement Tomorrow). This program was a partnership for 
workplace education among St. Louis Community College, Blue Cross and Blue Shield of 
Missouri (BCBSMo), and St. Louis Public Schools Adult Basic Education (ABE) Program. 



The primary goal of the effort was to counsel and train current BCBSMo workers with a 
need to upgrade and/or update their basic skills as part of the company's Selection System and 
related Selection Exercises. Mobility and advancement within the company are linked to 
successfully passing the Selection Exercises. The exercises address the three traditional skill 
areas of computation, reading, and writing, as well as the skills of perception, concentration, 
and visual memory. The STAT Program serves as the primary training support system for 
employees who are having difficulty passing all subtests of the exercises. The Selection 
Exercises are based on extensive job task analyses and measure skills needed to perform 
specific job functions at BCBSMo. 

While the STAT Program existed at BCBSMo prior to 1993, the federal grant allowed for 
significant expansion of services and the development and refinement of curriculum materials. 
Through the grant, STAT offered the following free educational services for BCBSMo 
employees: 

• Get ready for college courses 

• Brush-up in writing, math, and other basic business skills 

• Preparation for taking or re-taking the Selection Exercises 

• Individual.tutoring 

• Take-home study packets 

• Educational counseling 
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Classes and workshops were offered during the lunch hour, after work, and some 
Saturday mornings. STAT classes and workshops included the following: 

• Reading Skills Brush-Up 

• Math Skills Brush-Up 

• Punctuation and Grammar Skills 

• Business Writing Skills 

• Effective Communication Skills 

• Word Power 

• Improve Your Memory 

• Improve Your Concentration 

• Problem Solving 

• Study Skills and Test Anxiety 

Self study packets include Business Letter-Writing Exercises, Perception/Concentration 
Skills, Comprehension /Problem Solving, and Memory Skills. Grant-related services have been 
provided to a total of (515) participants. 

The grant provided support for an external evaluator to conduct formative and 
summative program evaluations. The purposes of the formative evaluation were to collect, 
analyze, and report data that would allow developers to improve the program design. In this 
context, major program variables were studied to determine whether or not the program was 
being implemented as planned, and how the program could be strengthened. The purposes of 
the summative evaluation were to collect, analyze, and report data on project worth. 
Important evaluation questions included: To what extent and in what ways do individual 
workers benefit from project participation? To what extent and how does BCBSMo benefit 
from the project? Do these benefits, if any, justify continuing the project? 



Program Development 

• Participant and instructor evaluations of STAT courses and workshops indicated 
that participants gave high ratings to the training activities and value the 
opportunities provided by the program. This was reinforced by classroom 
observations, participant responses to the evaluation questionnaires, and comments 
made by focus group participants. 

• Classes observed were extremely well planned and well received. Objectives were 
clear and there was a high level of interest, participation, and attention to task on 
the part of participants. All instructors were well prepared, warm, friendly, and 
accepting. 
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• Focus group participants found the classes helpful in developing new skills and 
preparing for the Selection Exercises. Instructors were highly rated, and the fact 
that they were all external to BCBSMo was seen as a strength. 

• Other perceived strengths of the training program include the following: flexible 
scheduling; small class size; voluntary attendance; availability of staff; various 
teaching styles; support from some managers for attending classes; support from 
classmates with similar needs; and individual assistance. 

• Three external and two internal reviewers reviewed the STAT curriculum for 
relevancy of the topics, quality of organization, appropriateness of reading level, 
and transferability. The reviewers gave high ratings to the materials. 



The program benefited individual workers in several ways, including increased skills, 
increased scores on the Selection Exercises, and increased self-confidence. 

• Participating workers showed significant increases in basic skills acquisition after 
completing STAT courses in reading, math, and punctuation /grammar. Based on 
pre-post gain scores: 95 percent improved their reading skills; 86 percent showed 
improvements in math skills; 81 percent improved their punctuation /grammar 
skills; and 85 percent showed overall increases in basic skills acquisition. 

• From 64 percent to 88 percent of STAT participants improved their scores on the 
Selection Exercises~88 percent improved their scores on the perception/ 
concentration section of the exercises, 80 percent improved their scores on the 
memory subtest, 70 percent improved their scores in numerical computation and 
written expression, and 64 percent improved in the area of comprehension /problem 
solving. 

• Sixty-three percent of STAT participants who re-took one or more sections of the 
Selection Exercises subsequently met company standards for continued 
employment and eligibility to compete for promotions. Furthermore, for the 
subtests of perception/concentration, written expression, and 
comprehension /problem solving, participants who completed a related STAT 
course before retaking one or more sections of the test received significantly higher 
gain scores than those who did not. 

• Ninety-two percent of participants felt that participation in the program had 
increased their self esteem; 60 percent indicated that it had increased their self- 
confidence to a great extent. This was reinforced by participant and supervisors 
comments. 
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Summative evaluation data suggest that some modest improvements in employee 
performance and productivity as well as increased customer satisfaction ratings may be 



attributable in part to program participation. 

• From 1991 to 1994, the percentage of STAT participants from claims and customer 
service whose performance ratings improved (49%) was considerably greater than 
the percentage whose scores declined (20%) or stayed the same (31%). 

• A considerably higher proportion of STAT participants from claims and customer 
service received improved productivity ratings (46%) over a four-year period as 
compared to those whose ratings declined (4%). Fifty percent of the ratings stayed 
the same from 1991 to 1994. 

• Over a four-year period, 42 percent of STAT participants sampled in the customer 
service department received improved customer satisfaction ratings, while none 
received lower ratings. The ratings for 58 percent stayed the same. 

According to BCBSMo administrators: 

• The project has made a difference at Blue Cross and Blue Shield of Missouri. It has filled a 
need to provide academic support to non-managerial staff, and had helped to change the 
nature and importance of employee input into company operations— the company will 
conduct employee climate surveys, all-employee meetings will be held, and employee 
feedback groups will be instituted. By the end of April 1995, every employee will have a 
development plan. 

• Employees need developmental support, want it, and STAT has played an importa7it role. 
This support needs to be provided into 1995 and beyond. It may become a more internal 
process in the future. 

Finally, the program has been institutionalized. Starting in January of 1995, STAT 
workshops will be offered on company time. The program has been restructured from a 
classroom instruction model to a more individualized, self-paced, computer-assisted approach 
provided in a multi-station basic skills lab. Blue Cross and Blue Shield of Missouri purchased 
all of the hardware for the lab and some of the software, and the grant supported the costs of 
additional basic skills software. The system is now being accessed by other Blue Cross and 
Blue Shield offices in Missouri. 
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I. INTRODUCTION AND BACKGROUND 



In the spring of 1993, the St. Louis Community College was awarded a National 
Workplace Literacy grant from the U.S. Department of Education to implement the STAT 
Program (Skills Today for Advancement Tomorrow). This program was a partnership for 
workplace education among St. Louis Community College, Blue Cross and Blue Shield of 
Missouri (BCBSMo), and St. Louis Public Schools Adult Basic Education (ABE) Program. 

The primary goal of the effort was to counsel and train current BCBSMo workers with a 
need to upgrade and/or update their basic skills as part of the company's Selection System and 
related Selection Exercises. Mobility and advancement within the company are linked to 
successfully passing the Selection Exercises. The exercises address the three traditional skill 
areas of computation, reading, and writing, as well as the skills of perception, concentration) 
and visual memory. The STAT Program serves as the primary training support system for 
employees who are having difficulty passing all subtests of the exercises. The Selection 
Exercises are based on extensive job task analyses and measure skills needed to perform 
specific job functions at BCBSM.3. The Human Resources Department sets related policies and 
administers the Selection Exercises. 

While the STAT Program existed at BCBSMo prior to 1993, the federal grant allowed for 
significant expansion of services and the development and refinement of curriculum materials. 
Through the grant, STAT offered the following free educational services for BCBSMo 
employees: 

• Get ready for college courses 

• Brush-up in writing, math, and other basic business skills 

• Preparation for taking or re-taking the Selection Exercises 

• Individual tutoring 

• Take-home study packets 

• Educational counseling 
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Classes and workshops were offered during the lunch hour, after work, and some 
Saturday mornings. STAT classes and workshops included the following: 

• Reading Skills Brush-Up 

• Math Skills Brush-Up 

• Punctuation and Grammar Skills 

• Business Writing Skills 

• Effective Communication Skills 

• Word Power 

• Improve Your Memory 

• Improve Your Concentration 

• Problem Solving 

• Study Skills and Test Anxiety 

Self study packets include Business Letter-Writing Exercises, Perception /Concentration 
Skills, Comprehension/Problem Solving, and Memory Skills. A STAT Course Catalog for the 
fall of 1993 is included as Appendix A. and illustrates the typical course offerings. 

Grant-related services have been provided 6,848 hours of instruction to a total of (515) 
participants. A breakout of participants by sex, race, age, years with the company, and hours 
in the program is provided in Table 1. 

The grant provided support for an external evaluator to conduct formative and 
summative program evaluations. An Interim Evaluation Report, summarizing formative 
evaluation activities and results, was submitted on May 12, 1994. This Final Evaluation Report 
summarizes the findings of the interim evaluation and presents the results of the summative 
evaluation. 

Section II of the report, Methodology, describes the formative and summative 
evaluation designs and related data collection strategies. Sections III and IV summarize the 
formative and summative evaluation results. Section V provides an overview of the findings. 
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Table 1 

STAT Participants by Sex, Race, Age, Years 
With the Company, and Hours in the Program 





Frequency 


%* 


Sex 




• 


Female 


448 


87% 


Male 


67 


13% 


Race 






White 


262 


51% 


African-American 


241 


47% 


Hispanic 


4 


1% 


Asian 


2 


.5% 


Native American 


2 


.5% 


Unknown 


4 


1% 


Age 






55+ 


21 


4% 


45-54 


103 


20% 


35-44 


210 


41% 


25-34 


171 


33% 


15-24 


5 


1% 


Years With the Company 






1-4 


104 


20% 


5-9 


149 


29% 


10-14 


73 


14% 


15-19 


81 


16% 


20-24 


74 


15% 


25-29 


23 


5% 


30+ 


6 


1% 


Hours in the Program 






1-5 


178 


35% 


6-9 


80 


16% 


10-14 


89 


17% 


15-19 


49 


10% 


20-24 


34 


7% 


25-29 


25 


5% 


30-34 


21 


4% 


35-39 


14 


3% 


40+ 


25 


5% 



*Percentages in each category will not always add up to 100% due to rounding. 
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II. METHODOLOGY 



Formative Evaluation 



The purposes of the formative evaluation were to collect, analyze, and report data that 
would allow developers to improve the program design. In this context, major program 
variables were studied to determine whether or not the program was being implemented as 
planned, and how the program could be strengthened. Program variables, formative 
evaluation questions, and formative assessment strategies are summarized in "Guidelines for 
Formative Evaluation," included as Appendix B. These guidelines were developed by the 
external evaluator, in consultation with and with the approval of, all members of the 
partnership. 

Data collection included: 

• Individual participant and instructor evaluations of each course and workshop 

• Observations of selected training activities 

• Interviews of selected BCBSMo administrators 

• Participant and supervisor responses to an interim project evaluation questionnaire 

• Focus groups with selected participants 

• Focus group with selected supervisors 



The following data collection instruments and formats are appended: 



Appendix C. Classroom Observation Form 

Appendix D. Interview Format 

Appendix E. Interim Project Evaluation Questionnaire 

Appendix F. Focus Group Guidelines and Discussion Form for Participants 

Appendix G. Focus Group Discussion Form for Supervisors 



Summativc Evaluation 



The purposes of the summative evaluation were to collect, analyze, and report date on 
project worth. Important evaluation questions included: To what extent and in what ways do 
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individual workers benefit from project participation? To what extent and how does BCBSMo 
benefit from the project? Do these benefits, if any, justify continuing the project? 

The summative evaluation design addressed each of the project's stated goals and 
performance objectives.* Specific evaluation questions included the following: 

• To what extent do individual workers benefit from participating in the STAT 
project? 

• To what extent does BCBSMo benefit from involving their workers in the STAT 
project? 

• To what extent do prospective workers benefit from participating in the STAT 
project? 

• To what extent does BCBSMo benefit from providing STAT services to prospective 
workers? 

• To what extent do the project's assessment and evaluation systems provide new 
measurement tools for the health care benefits industry? 

• To what extent are the project's curriculum materials valid, competency based, and 
transferable to similar settings? 

• To what extent are the workers involved in all aspects of the STAT project's 
development? 

• To what extent are project evaluation strategies designed jointly by STAT partners 
and the project evaluator? 

• To what extent is the program continued at the conclusion of the initial project 
period? 

Data collection included: 

• Pre-post gains on the Reading Brush-Up, Math Brush-Up, and Punctuation and 
Grammar tests; 

• Pre-post scores on the BCBSMo Selection Exercises (five subtests and an overall 
average score); 



1 Evaluation activities related to goals 2 and 5 were eliminated. The purpose of goal 2 was to develop a 
recruitment and training referral system wherein prospective workers would be trained to successfully 
compete for jobs at BCBSMo. Because of a reorganization and hiring freeze, there was no longer a need 
for a pool of potential workers. Goal 5, to involve workers in all aspects of ST ATs program 
development, was not implemented due to confidentiality issues. 
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• Supervisor performance appraisals for each of four years for a sample of 
participants working in the claims and customer service departments; 

• Job retention, attendance, punctuality, productivity, and quality of work (error rate 
and customer satisfaction) ratings, collected over a period of four years, for a sample 
of participants working in the claims and customer service departments; and 

• Perceptions of all members of the partnership with regard to program strengths, 
weaknesses, and future plans. 

Project goals, performance objectives, evaluation questions, evaluation criteria, and 
assessment strategies are summarized in "Guidelines for Summative Evaluation," included as 
Appendix H. These guidelines were developed by the external evaluator, in consultation with 
all members of the partnership. 
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III. RESULTS OF THE FORMATIVE EVALUATION 



Results of the following assessment components are presented and discussed in this 
section of the report 

• Individual Course/Workshop Evaluations 

• Classroom Observations 

• Interviews 

• Interim Project Evaluation Questionnaire 

• Focus Groups 

Participant evaluation forms were completed for most of the STAT courses and 
workshops. Completed forms were reviewed by program staff at the end of each training 
event, and results were used to refine training schedules, presentation strategies, and materials. 
In general, participants gave the courses and workshops very high ratings and have valued the 
opportunities provided by the program. Results of a course-by -course analysis, for each of six 
evaluation questions, is presented in Tables 2 through 7. 



Table 2 

STAT Course/Workshop Evaluations 



To what extent zvas the session instructive and useful? 

Not Helpful Very Helpful . 

1 2 3 4 5 6 7 



Course/Workshop 


N 


Range of 
Responses 


Mean 


Perception/Concentration 


55 


4-7 


6.10 


Punctuation & Grammar 


44 


4-7 


6.48 


Math 


60 


5-7 


6.77 


Communication 


56 


5-7 


6.54 


Memory 


36 


5-7 


6.69 


Power Reading 


25 


5-7 


6.64 


Drawing Conclusions 


35 


4-7 


5.89 


Word Power 


53 


5-7 


6.72 


Study Skills 


21 


4-7 


6.14 


Business Writing 


18 


4-7 


6.11 
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Table 3 

STAT Course/ Workshop Evaluations 



How was the pace of the workshop (i.e., the information covered compared to the 
, time spent covering it)? 

: Too Slow Too Fast 

■ 1 ■ .'. 2 3 4 5 6 7 



Course / Workshop 


N 


Range of 
Responses 


Mean 


Perception/Concentration 


55 


3-7 


4.80 


Punctuation & Grammar 


43 


4-7 


5.05 


Math 


57 


3-7 


5.44 


Communication 


61 


4-7 


5.87 


Memory 


38 


4-6 


4.76 


Power Reading 


25 


4-6 


4.60 


Drawing Conclusions 


35 


4-7 


5.19 


Word Power 


53 


4-7 


5.12 


Study Skills 


20 


4-6 


4.80 


Business Writing 


18 


3-7 


5.22 
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Table 4 

STAT Course/Workshop Evaluations 



. Please evaluate the instructor(s) on their knowledge of the subject. 

Poor Excellent 
: 2 2 3 4 5 6 7 



Course / Workshop 


N 


Range of 
Responses 


Mean 


Perception/Concentration 


55 


4-7 


6.49 


Punctuation & Grammar 


44 


5-7 


6.80 


Math 


60 


4-7 


6.78 


Communication 


61 


5-7 


6.62 


Memory 


39 


6-7 


6.85 


Power Reading 


25 


6-7 


6.76 


Drawing Conclusions 


35 


4-7 


5.63 


Word Power 


53 


5-7 


6.83 


Study Skills 


21 


4-7 


6.57 


Business Writing 


18 


5-7 


6.28- 
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Table5 

STAT Course/Workshop Evaluations 



Please evaluate the instructor(s) on their presentation skills. 

Poor Excellent 
1 2 3 4 5 6 7 



Course/Workshop 


N 


Range of 
Responses 


Mean 


Perception /Concentration 


55 


4-7 


6.67 


Punctuation & Grammar 


44 


6-7 


6.75 


Math 


60 


4-7 


6.70 


Communication 


62 


5-7 


6.52 


Memory 


39 


6-7 


6.85 


Power Reading 


25 


6-7 


6.80 


Drawing Conclusions 


35 


3-7 


6.09 


Word Power 


53 


5-7 


6.79 


Study Skills 


21 


5-7 


6.71 


Business Writing 


17 


5-7 


6.30 
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Table 6 

STAT Course/Workshop Evaluations 



Please evaluate ihe instructor(s) on their response to questions. 

Poor Excellent 
1 2 3 4 5 6 7 



Course/Workshop 


N 


Range of 
Responses 


Mean 


Perception/Concentration 


55 


4-7 


6.67 


Punctuation & Grammar 


44 


6-7 


6.86 


Math 


60 


4-7 


6.78 


Communication 


63 


5-7 


6.52 


Memory 


39 


6-7 


6.85 


Power Reading 


24 


6-7 


6.92 


Drawing Conclusions 


35 


4-7 


5.63 


Word Power 


53 


6-7 


6.91 


Study Skills 


21 


5-7 


6.71 


Business Writing 


18 


6-7 


6.56 
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Table 7 

STAT Course/Workshop Evaluations 



Would you recommend this or a similar course to your co-workers? 
Yes No 



Course/Workshop 


N 


% Responding "Yes" 


Perception/Concentration 


55 


89% 


Punctuation & Grammar 


44 


89% 


Math 


60 


97% 


Communication 


64 


91% 


Memory 


39 


80% 


Power Reading 


25 


84% 


Drawing Conclusions 


35 


91% 


Word Power 


53 


94% 


Study Skills 


21 


95% 


Business Writing 


18 


94% 
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Three different courses and one workshop were each observed for a brief period of 15 to 
30 minutes. Originally, the evaluator planned to spend much more time observing training 
activities. This strategy was dropped after the initial four observations because it became 
obvious that the sessions were extremely well planned, and extremely well received by 
participants. 

The four training activities observed were: 

• Math Skills Brush Up (Section 1) 

• Math Skills Brush Up (Section 2) 

• Punctuation & Grammar 

• Improving Your Memory 

A copy of the Classroom Observation Form is included as Appendix C. Results of the 
observations are summarized below. 

Clarity of Objectives. In two of the sessions observed, the evaluator was present at the 
beginning of the session. In both cases, the instructor clearly stated the objectives of the 
sessions and what he or she hoped to accomplish during the class period. In the other 
two sessions, the evaluator arrived while the classes were already in progress. 
Objectives of each of these training sessions also seemed to be clear, given that 
participants were following along without any apparent confusion. 

Level of In terest. Participation, and Attention to Task . Levels of interest, participation, 
and attention to task were high in all classes observed. This is not surprising in a 
voluntary program, particularly when the instructors are very effective. 

Quality of Teaching Methods and Instructional Materials . Four different instructors 
were observed. The two math instructors used somewhat different teaching styles. 
One used a direct presentational approach. The other used a more informal, group 
problem-solving approach. Students in the latter group were placed in the class 
because of a high level of math anxiety. All instructors were well prepared, warm, 
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friendly, accepting, and apparently helpful. The instructional materials were not 
examined closely but appeared to be clear and well organized. 



Three BCBSMo administrators were interviewed, two of whom are directly involved in 
the administration of the STAT Program. The purpose of these interviews was to get a very 
general reading of BCBSMo's perceptions of program impact on the company and needs for 
improvement. Comments are summarized below. 

1. The addition of workplace literacy resources provided to the program through the 
federal grant and related three-way partnership has benefited the company and 
participants in a variety of ways: 

• The scope of the program was increased. 

• Additional resources and expertise were applied to the program, thereby 
reducing pressure on BCBSMo staff. 

• Additional community resources were identified and focused on the 
program. 

2. Benefits to parjieipants have included the following: 

^ • The program has had a tremendous confidence-building effect. Some 

participants were actually talking about going to college. Once they were 
on a "learning track," they wanted to stay on it. 



• Participants have developed confidence and are learning how to learn. 
This has in turn helped morale and productivity, and ultimately is in the 
best interest of the company. 

3. Overall strengths of the training program include the following: 

• The fact that instructors were professional educators external to the 
company made participating workers feel more comfortable. "You don't 
feel stupid when it's a teacher." 

• Instructors have been extremely sensitive to workers' needs and 
concerns. 

• The volunteer nature of the program has worked well. 
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• There has been a tremendously positive reaction from employees. 



4. BCBSMo administrators expressed some disappointment that the external 

component had not worked as planned. They explained that the employment 
needs of the company changed over the last few years, and an external pool of 
qualified applicants was less critical than it had been previously. 



At the end of the grant period, the program will be restructured from a 
classroom approach to individualized, self-paced, computer-assisted instruction 
provided in a skills center. 



_ Inter im "Project Evaluation Questionxtaire ; 



In mid-February, 334 current and past STAT participants and 55 BCBSMo supervisors 
were asked to complete brief questionnaires regarding their experiences with STAT Program 
services. Completed forms were received from 79 participants and 12 supervisors. Participant 
and supervisor responses are summarized in Tables 8 and 9. Additional suggestions and 
comments are included as Appendix I. 
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Table 8 

Participant Responses: Interim Project Evaluation Questionnaire 



N" 



Mean* 



SD 



Question 1: To what extent are STATs goals 77 
and objectives clear? 

Question 2: To what extent is STAT well 77 
s organized? 

Question 3: To what extent are recruitment 71 
strategies effective? 

Question 4: To what extent do training 76 
activities meet real needs? 

Question 5: To what extent are instructors, 75 
instructional methods, and 
materials of high quality? 

Question 6: To what extent are training 76 
activities scheduled at convenient 
times? 

Question 7: To what extent are training 73 
activities successful in improving 
workplace literacy skills? 

Question 8: To what extent are STAT support 74 
services, such as the Skills Center, 
self-study exercises, etc., helpful to 
participants? 

Question 9: To what extent have you taken 76 
advantage of these services? 

Question 10: To what extent have you found 74 
these services helpful? 

Question 11: To what extent has your participation 75 
in STAT increased your self confidence? 

Question 12: To what extent has your participation 75 

in STAT helped you to become a 
better worker? 



4.09 



4.16 



3.77 



3.88 



4.21 



3.54 



4.11 



4.23 



3.72 



4.19 



3.80 



3.68 



.80 



.73 



.93 



.82 



.81 



1.06 



.72 



.63 



.92 



.73 



.99 



1.04 



* Scored on a five-point scale: 5 = To a Great Extent; 1 = Not at All. 
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Table 9 

Supervisor Responses: Interim Project Evaluation Questionnaire 



Mean* 



SD 



Question 1: To what extent are you aware 12 
of the STAT project? 

Question 2: To what extent are STATs goals 12 
and objectives clear? 

Question 3: To what extent is STAT well 11 
organized? 

Question 4: To what extent do STAT staff 10 
seem to be well qualified? 

Question 5: To what extent are recruitment 10 
strategies reaching the BCBSMo 
workers for whom they are intended? 

Question 6: To what extent are training 9 
activities successful in improving 
workplace literacy skills? 

Question 7: To what extent are other STAT 9 
support services, such as the Skills 
Center, self-study exercises, etc., 
by STAT helpful to participants? 



3.67 



3.67 



3.55 



4.00 



3.20 



3.55 



4.11 



1.07 



.89 



.52 



.82 



.79 



.73 



.60 



* Of the 12 respondents, not every supervisor answered every question. 
** Scored on a five-point scale: 5 = To a Great Extent; 1 = Not at All. 
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Three focus groups were facilitated by the external evaluator-two with groups of STAT 
participants and one with supervisors. 



Focus Group With STAT Participants 

Focus group participants were selected randomly. Of the 369 individuals the program 
has served through the federal grant to date, 20 names were randomly pulled from the files. 
These individuals were contacted and asked to attend one of two focus groups. Of the 20 
individuals contacted, 12 were able to attend one of the groups. 4 

The first focus group, held April 18 at the Chestnut location, was attended by six 
workers. These individuals had participated in from two to eight STAT courses or workshops. 
Participants' comments and suggestions are summarized below. 

Strengths /Benefits 

• New skills gained from the Punctuation/Grammar and Writing courses 

• Changed perceptions of ability 

• Preparation for selection exercises 

• Improved scores on the Selection Exercises 

• Class schedule-flexible scheduling, classes during the lunch hour and after work 

• Voluntary nature of the program 

• Availability of staff 

• Instructors external to BCBSMo-it is easier to be honest with them 

• Instructors are non-judgmental, trained to listen, seemed to care more 

• Instructors help workers examine options 

• Various teachers with different teaching styles 

• Small class size 

• 1:1 assistance 

• Support from classmates with similar problems /needs 

• Some managers encourage workers to attend classes 

• Short-term training; not locked in forever 

• Option to take classes many times 

• No cost 



We aknesses /Recommendations 

• Focus of learning is on the test rather than the skills 

• Intimidation of the testing situation 

• Need more test-taking practice 

• Information on program wasn't readily available, particularly at City Center 
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The second focus group, held at One City Center on April 21, was also attended by six 
workers. Their experience with the program ranged from two to eight courses or workshops. 
All had taken part in at least one home study exercise. Participant comments and suggestions 
follow. 

Strengths /Benefits 

• Keeping the job 

• Promotion potential 

• Preparation for the Selection Exercises 

• Probably helps with on-the-job skills, but not as obvious 

• Refreshed skills, personal and on-the-job 

• Instructors made you f^el comfortable 

• Instructors were sensitive and perceptive 

• Availability of instructors 

• 1:1 training 

• The home study option allows for individualized pacing and 1:1 assistance later 

• Lunch hour classes* 

• Some supervisors encourage attendance 

Weaknesses / Recommendations 

• Lunch hour classes* 

• Not enough time in. classes for discussion or help 

• Going to after-work classes at another site 
. • Resistance from some supervisors 

• Last year's materials were weak and had numerous mistakes 

• Some of the STAT mailings weren't received at all or weren't received in time to 
respond 

• Some courses/workshops by themselves aren't that helpful in building skills 
because more individual practice is needed 

• Need a class on office etiquette for everyone 

* One participant felt that lunch hour classes were a strength of the program; another felt this 
was a weakness. 



Focus Group With Supervisors 

A focus group for supervisors was held at the Chestnut location on October 12, 1994, 
from 8:30 to 9:30. Eleven supervisors attended the session. These supervisors were randomly 
selected from all supervisors across the company. Participating supervisors were asked to 
respond to six discussion questions. These questions, along with supervisor responses, are 
summarized on the next page. 
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1. How familiar are you with the STAT Prog ram? 

Of the 11 participants, two reported being very familiar with STAT; the remaining nine said 
they were at least somewhat familiar with the program. 

2. Have any of your employees participated in STAT ? 

Nine of the participants knew for sure that employees under their supervision had participated 
in STAT; one said she didn't supervise any STAT participants, and the remaining supervisor 
said she didn't know whether or not any of her employees had participated in the program. 

3. In what ways do yoti think employees have benefited from the STAT Program? 

• Job retention 

• Promotion 

• Increased motivation to pursue further education, sometimes even college 

• Increased skills 

• Increased self-confidence 

4. To what extent has STAT met your needs as a supervisor ? 

Supervisors felt that the program was not really meeting their needs because they were not in 
the information loop. They expressed a desire to know more about the program, including 
which of their employees were involved and how these individuals were benefiting from 
participation. 

5. What are some of the greatest strengths of the program ? 

• Training of employees 

• Building confidence 

• Flexibility in meeting individual needs 

• Voluntary nature of the program 

• Home study option 

6. In what ways could the program be improved ? (and other suggestions) 

Include supervisors/managers in the information loop 
Provide supervisors /managers with feedback on program progress 
Provide managers with an overview of skills taught 
Include an overview of STAT as part of the orientation for new employees 
Market the program as a vehicle for personal/career development rather than 
remediation 

Change the project name to remove the association with remediation 
Separate professional development from selection exercises 
Provide incentives for participation 

Advertise program and individual accomplishments-accomplishment board, 
testimonials from participants, newsletter 

Provide a second level of skills upgrades for employees above the clerical level 
Reach production workers 
Videotape classes for home study use 
Continue the use of practice tests 

Provide classes in sales, communication, rime management, organizational skills 
Have brown bag lunches 
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IV. RESULTS OF THE SUMMATIVE EVALUATION 



Information on project worth is reported in the following four sections of this report: 



• Benefits to Individual Workers 

• Benefits to BCBSMo 

• Development of Program Models 

• Shared Perceptions of Partner Institutions 



Benefits to Individual Workers 



Development of Basic Skills 



Well over 70 percent of participants showed an increase in basic skills acquisition after 
taking courses in reading, math, and punctuation /grammar. Based on their pre-post gain 
scores: 95 percent improved in the area of reading; 86 percent improved in the area of math; 81 
percent improved in punctuation /grammar; and 85 percent showing an overall increase across 
the three content areas. Mean pre- and post-test scores in each area are shown in Table 9. 



Table 9 

Mean Pre- and Post-test Scores in Reading, Math, and Punctuation /Grammar* 



Content Area 


N 


Mean Pre-test Score 


Mean Post-test Score 


Reading 


40 


13.60 


17.88 


Math 


86 


12.50 


18.34 


Punctuation/ 
Grammar 


89 


38.79 


43.91 



•Number possible: reading = 22; math = 30; punctuation/grammar = 55. 
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Overall Improvement of Scores on the 
Selection Exercises 

As illustrated in Table 10, a high percentage of STAT participants improved their scores 
on the Selection Exercises. Eighty-eight percent (88%) who re-took the perception/ 
concentration section of the exercises improved their scores, 80 percent improved their scores 
on the memory subtest, 70 percent improved their scores in numerical computation and 
written expression, and 64 percent improved in the area of comprehension/problem solving. 



Table 10 

Percent of STAT Participants Who Improved Their Scores on the Selection Exercises 



Subtest 


Number Re-taking 
the Subtest 


Number Improving 
Their Scores 


Percent Improving 
Their Scores 


Perception/ 
Concentration 


130 


114 


88% 


Memory 


74 


59 


80% 


Numerical 
Computation 


111 


78 


70% 


Written Expression 


79 


55 


70% 


Comprehension/ 
Problem Solving . 


89 


57 


64% 



Eligibility to Compete for Promotions 

While the projected 70 percent criterion was not reached, 63 percent of STAT 
participants who re-took one or more sections of the Selection Exercises subsequently met 
company standards for continued employment (passing scores on each subtest and the overall 
score) and eligibility to compete for promotions. Furthermore, for the subtests of 
perception/concentration, written expression, and comprehension/problem solving, 
participants who completed a related STAT course before retaking one or more sections of the 

test received significantly higher gain scores than those who did not (see Table 11). 
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Table 11 

Mean Gain Scores on Selection Exercises for Individuals Who Completed 
a Relevant STAT Course/Workshop vs. Those Who Did Not 



Courses/Workshops 


N 


Mean Gain Score* 


Standard Deviation 


Perception/ 
Concentration 








Completed 
a STAT course 


94 


2.17** 


1.35 


Did not complete 
a STAT course 


37 


1.54** 


1.35 


Memory 








Completed 
a STAT course 


38 


1.97 


1.68 


Did not complete 
a STAT course 


36 


2.33 


1.88 


Numerical 
Computation 








Completed 
a STAT course 


93 


1.54 


1.44 


Did not complete 
a STAT course 


17 


.88 


1.11 


Written Expression 








Completed 
a STAT course 


66 


1.45** 


1.37 


Did not complete 
a STAT course 


13 


.38** 


1.26 



* Scored on a six-point scale: 6 = Superior; 1 = Unsatisfactory. 



**p < .02 
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Table 11, continued 
Mean Gain Scores on Selection Exercises for Individuals Who Completed 
a Relevant STAT Course/Workshop vs. Those Who Did Not 



Courses / Workshops 


N 


Mean* 


Standard Deviation 


Comprehension/ 
Problem Solving 








Completed 
a STAT course 


64 


1.25** 


1.45 


Did not complete 
a STAT course 


25 


.42** 


1.25 


Overall Score 

(mean for all five 
subtests) 








Completed 
a STAT course 


131 


.62 


.69 


Did not complete 
a STAT course 


98 


.75 


.68 



* Scored on a six-point scale: 6 = Superior; 1 = Unsatisfactory. 
**p<.02 



Job Retention 



A job retention study of 105 employees in BCBSMo's customer service and claims 
departments was designed and conducted by program staff. Results for each department are 
presented in Tables 12 and 13. 
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Table 12 

Job Retention Data for Individuals Who Participated in STAT 
vs. Those Who Did Not: Customer Service Department* 





STAT 

(1 or more hours) 


No STAT 


Total Number 


Kept Job 


19 


9 


28 


Lost Job 


4 


9 


13 


Total 


23 


18 


41 



*Note: In December of 1993, the Customer Service Department (41 employees) needed to 
downsize due to reorganization. Thirteen service representative positions were marked for 
elimination. Customer Service management and the Human Resources Department used the 
following criteria to determine which 13 employees would be downsized: 1992-93 performance 
review ratings; formalized counselings (for problems on the job); Selection Exercise scores; 
Writing Skills Exercise scores; and attendance. 



Table 13 

Job Retention Data for Individuals Who Participated in STAT 
vs. Those Who Did Not: Claims Department* 





STAT 

(1 or more hours) 


No STAT 


Total Number 


Kept Job 


21 


27 


48 


Lost Job 


8 


8 


16 


Total 


29 


35 


64 



*Note: In December 1993, the Claims Department needed to downsize its professional 
evaluator staff (64) by 16 positions. Of these 16 employees, four were terminated and twelve 
were placed in lower level positions in a different department. Criteria for determining which 
16 employees would be downsized included the following: 1992-93 performance review 
ratings; formalized counselings (for problems on the job); Selection Exercise scores; and 
attendance. 
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It appears that STAT participation may have been a factor in job retention in the 
Customer Service Department, however, many other variables were also taken into account as 
indicated in the Note following Table 12. 

Self-confidence and Empowerment 

Participant responses to question #11 of the Interim Evaluation Questionnaire indicate 
that 92 percent of participants felt that participation in the program had increased their self- 
confidence; 60 percent indicated that it had increased their self-confidence to a great extent. 
This was reinforced by participant and supervisors comments in the focus groups conducted as 
part of the formative evaluation. 

i Benefits to BCBSMo '" = ; " ;:. •<.'•" :;<0^^1» >. 

Performance Appraisals 

Table 14 illustrates mean supervisor performance ratings over a four-year period for a 
sample of STAT participants. The sample represents BCBSMo's customer service and claims 
departments. While the means increase over the four-year period, the increases are small and 
can't necessarily be attributed to STAT participation. The performance ratings were assigned 
by supervisors during annual performance reviews of all company employees. The ratings are 
highly subjective, and in any four-year period for a single employee, performance may have 
been rated by as many as four different supervisors. 

Table 15 shows the percentage of STAT participants whose performance appraisals 
showed improvement, stayed the same, or declined over this same four-year period and over 
the last year. For the four-year comparison ranging from 1991 to 1994, the percentage of STAT 
participants whose performance ratings improved was considerably greater than the 
percentage whose ratings declined. This did not, however, reach the 70 percent level as hoped. 
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Table 14 

Mean Supervisor Performance Ratings for a Sample of 
STAT Participants Over a Four-year Period 



Year of Rating 


N* 


Mean** 


Standard Deviation 


1990-1991 


55 


2.91 


.67 


1991-1992 


68 


3.03 


.65 


1992-1993 


69 


3.22 


.59 


1993-1994 


67 


3.21 


.57 



* Numbers represent STAT participants who held the relevant position during the year of 
rating. 



** Four-point rating scale: 4=consistently surpasses standards; 3=consistently meets standards; 
2=usually meets standards; and l=does not meet standards. 



Table 15 

Percentage of a Sample of STAT Participants Whose Performance Appraisals Showed 
Improvement, Stayed the Same, or Declined Over a Four-year Period 



Years 


N 


% Improved 


% Stayed Same 


[ % Declined 


1991-1994 


49 


49% 


31% 


20% 



Productivity 
Productivity Ratings 

Table 16 illustrates mean productivity ratings for STAT participants sampled from the 
customer service and claims departments. The ratings are determined by supervisors and 
pertain to the number of customers or claims handled. According to program staff, 
supervisors cautioned them not to give too much credence to these ratings because the 
standards often change for year to year, and reorganization may result in fewer customer 
service reps taking more calls. However, as illustrated in Table 17, a considerably higher 
proportion of STAT participants in the sample received improved productivity ratings (46%) as 
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compared to those whose ratings declined (4%) over the four-year comparison period. The 
same holds for the one-year comparison. 



Table 16 

Mean Productivity Ratings for a Sample of 
STAT Participants Over a Four-year Period 



Year of Rating 


N* 


Mean** 


Standard Deviation 


1990-1991 


55 


2.96 


.84 


1991-1992 


67 


3.12 


.73 


1992-1993 


65 


3.09 


.74 


1993-1994 


66 


3.33 


.71 



* Numbers represent STAT participants who held the relevant position during the year of 
rating. 



** Four-point rating scale: 4=consistently surpasses standards; 3=consistently meets standards; 
2=usually meets standards; and l=does not meet standards. 



Table 17 

Percentage of a Sample of STAT Participants Whose Productivity Ratings Showed 
Improvement, Stayed the Same, or Declined Over a Four-year Period 



Years 


N 


% Improved 


% Stayed Same 


% Declined 


1991-1994 . 


48 


46% 


50% 


4% 



Attendance 

Attendance data for the sample of STAT participants is shown in Tables 18 and 19. The 
data are inconclusive as to what relationship program participation had on attendance. An 
experimental-control comparison would be more useful in examining this relationship. 
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Table 18 

Mean Attendance Figures for a Sample of 
STAT Participants Over a Four-year Period 



Year of Rating 


N* 


Mean# of Days 
Absent 


Standard Deviation 


1990-1991 


50 


3.06 


.27 


1991-1992 


57 


4.25 


.90 


1992-1993 


64 


4.20 


.83 


1993-1994 


61 


3.20 


.45 



* Numbers represent STAT participants who held the relevant position during the year of 
rating. 



Table 19 

Percentage of a Sample of STAT Participants Whose Attendance Showed 
Improvement, Stayed the Same, or Declined Over a Four-year Period 



Years 


N 


% Improved 


% Stayed Same 


% Declined 


1991-1994 


41 


44% 


22% 


34% 



Punctuality 

Punctuality ratings are shown in Tables 20 and 21. Like the attendance date, little 
useful information can be gained regarding the program's impact on punctuality. 
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Table 20 

Mean Punctuality Figures for a Sample of 
STAT Participants Over a Four-year Period 



Year of Rating 


N 


Mean** 


Standard Deviation 


1990-1991 


55 


2.47 


.50 


1991-1992 


68 


2.43 


.55 


1992-1993 


69 


2.23 


.46 


1993-1994 


65 


2.38 


.55 



* Numbers represent STAT participants who held the relevant position during the year of 
rating. 



** Three-point rating scale: 3=particular strength; 2= no comment necessary; l=needs 
improvement. -■■ 



Table 21 

Percentage of a Sample of STAT Participants Whose Punctuality Showed 
Improvement, Stayed the Same, or Declined Over a Four-year Period 



Years 


N 


% Improved 


% Stayed Same 


% Declined 


1991-1994 


47 


11% 


60% 


30% 



Quality of Work 

Quality ratings of employees' work were also collected for a sample of STAT employees 
in the customer service and claims departments. For the claims department, qualitative ratings 
were based on error rate. In customer service, ratings were based on customer satisfaction. 
Table 22 illustrates mean qualitative ratings over a four-year period. The differences are too 
small to warrant any conclusions about program effects. However, as shown in Table 23, over 
a four-year period from 1991-1994, 42 percent of STAT participants sampled in the customer 
service department received improved customer satisfaction ratings, while none received lower 
ratings. 
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Table 22 

Mean Qualitative Ratings for a Sample of 
STAT Participants Over a Four-year Period 



Year of Rating 


N* 


Mean** 


Standard Deviation 


1990-1991 


54 


2.93 


.75 


1991-1992 


65 


2.97 


.73 


1992-1993 


68 


3.26 


.66 


1993-1994 


65 


3.22 


.54 



* Numbers represent STAT participants who held' the relevant position during the year of 
rating. 

** Four-point rating scale: 4 = Consistently Surpasses Standards; 1 = Does Not Meet Standards. 



Table 23 

Percentage of a Sample of STAT Participants in Customer Service Whose 
Customer Satisfaction Ratings Showed Improvement, Stayed the Same, 
or Declined Over a Four-year Period 



Years 


N 


% Improved 


% Stayed Same 


% Declined 


1991-1994 


19 


42% 


58% 


0% 



Development of Program Models 

Assessment and Evaluation Systems 

A formal, external analysis of program assessment and evaluation systems was not 
conducted as planned because the evaluation system is still in a developmental stage. 

C ompetency-based Curriculum 

Three external reviewers and two internal reviewers were asked to review the following 
curriculum packets: Perception/Concentration; Comprehension /Problem Solving; Memory 
Skills; Math; and Writing. Each packet was reviewed for relevancy of the topic, quality of 
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organization, appropriateness of reading level, and transferability. Copies of the curriculum 
analysis forms are included as Appendix J. The results of these analyses is summarized in 
Table 24. In general, the reviewers gave high ratings to the materials. 

Table 24 



Results of Curriculum Analysis for Perception/Concentration, 
Comprehension /Problem Solving, Memory Skills, Math, and Writing 



*wUrncuium /\rea/ rvaung 


iviean ivaung 


XNumoer or reviewers 


Category 






r ercepuon / v^oncentration 




3 external reviewers 


Keievancy or lopic 






v/rganiza uon 


4. DO 




Reacting Level 


A K/i 




Transferability 


4.22 




v^uinprenciisiuii/ r ruuicin 




o external reviewers 


DOlVlIlg 






i\eievancy or lupic 


A KT\ 




wrgaii Lea nun 


A 33 




Reading Level 


5.00 




X 1 CU CXXJLLli. y 


4 35 




iviemory dkiiis 




3 external reviewers 


iveievancy oi lopic 


A A7 
I.Ol 




Organization 


4.50 




Reading Level 


4.33 




Transferability 


3.50 




Math 




1 internal reviewer 


Relevancy of Topic 


4.00 




Organization 


4.00 




Reading Level 


4.00 




Transferability 


4.00 




Writing 




1 internal reviewer 


Relevancy of Topic 


4.00 




Organization 


4.29 




Reading Level 


4.00 




Transferability 


4.14 





* Scored on a five-point scale: 5 = High; 1 = Low. 



A meeting of representatives of all three STAT partners was held on December 6, 1994. 
A copy of the meeting agenda and meeting summary are included as Appendix K. Four major 
topics covered were: review of project outcomes; shared perceptions of the project by each 
partner; future partnership activities and collaboration; and dissemination. 

Following a brief overview of the evaluation findings to date provided by the external 
evaluator, partners discussed their perceptions of what worked, what didn't work, and what 
they would do differently. The following comments and suggestions were generated 1 : 

• The project has made a difference at Blue Cross and Blue Shield of Missouri. It has filled a 
need to provide academic support to non-managerial staff, and had helped to change the 
nature and importance of employee input into company operations— the company xvill 
conduct employee climate surveys, all-employee meetings mil be held, and employee 
feedback groups will be instituted. By the end of April 1 995, every employee mil have a 
development plan. (BCBS) 2 

• Some things have worked marvelously well, some things haven 't worked as well. If we had a 
chance, we'd do it again. Could we have done more? Probably, but we didn't. (BCBS) 

• The project was key to helping employees get through a period of reorganization and 
downsizing, and it gave them something they could do about their situations. (BCBS) 

• Employees need developmental support, want it, and STAT has played an important role. 
This support needs to be provided into 1995 and beyond. It may become a more internal 
process in the future. (BCBS) 

• The program has been restructured from a classroom instruction model to a more 
individualized, self-paced, computer assisted approach provided in a skills center. In 
addition, starting in January of 1995, STAT workshops will be offered on company time. 
(BCBS) 

• A strength of the project from the perspective of the educational providers was that it was 
already in place in the company, and additional job audits weren't required. (SLCC) 3 

• Targeting services and focusing on a single department [at a time], rather than trying to 
serve everyone [at the same time], might have been more likely to lead to measurable 
impact. However, the company had a need to provide services company wide. (SLCC) 



1 These comments and suggestions are exact or nearly exact quotes from meeting participants. 

2 BCBS-Alliance Blue Cross and Blue Shield of Missouri 

3 St. Louis Community College 
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• Next time we might narrow our expectations. (SLCC) . , 

• While peer involvement in project planning was built into the proposal, it didn't work 
because the culture wasn't right. The educational provider needs to be flexible enough to 
respond to the business culture. (SLCC) 

• It was hoped that the curriculum could be computerized before the end of the project, but that 
didn't happen. (SLCC) 

• The project would have benefited from much more interface and integration with managers 
and supervisors. (SLCC) 

• The external component, designed to train potential workers, didn't get off the ground for a 
number of reasons, including a reorganization and hiring freeze at Blue Cross and Blue 
Shield that wasn't anticipated when the grant was initially submitted. Other problems 
encountered may have been related to Missouri's ABE structure. If this component had 
been implemented, it could have had some positive spin-offs, including benefits to the 
community. (ABE) 4 

• [From the perspective of the ABE Program Director] the project was another 
opportunity to broaden the horizons of some clients, but not directly related to employability 
in the community. It is difficult to related general training materials to employability. 
Clients want something that is tied to getting a job [a concrete intervention and a 
definite connection to actual jobs]. (ABE) 

• Today, workers are in transition, and worker training needs to be a lifelong thing. This is 
finally receiving some attention at the federal and state levels. (BCBS) 

• One of the most important questions for employers to consider with respect to employee 
development is "Does the individual continue to grow in the company and demonstrate a 
continuing willingness and ability to learn?" (SLCC) 

• Approaches to workforce literacy should focus on the ideas of developing workers who can 
continue to learn, rather than on just, improving basic skill deficits. (SLCC) 

• The project needs to be more visible in the community. (BCBS) 

• We can serve as resources for one another, and we need to work together collaboratively in 
the community, and do our own PR work. (BCBS) 

• We will continue to utilize both of the education partners as providers of on-site skills 
enhancement. (BCBS) 

• The community needs to be educated about the broader definition of literacy today; it's not 
just reading and writing anymore. Workers need to have skills that will allow them to adapt 
to a changing workplace and become lifelong learners. (SLCC) 



4 St. Louis Public Schools Adult Basic Education Program 
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There is a paucity of understoiding about adult literacy in St. Louis companies. It isn't 
perceived as a high need. It's not clear what will cause the business community to wake up. 

(bcbs; 

The role of adult basic education in the workplace is not clearly defined. From the 
perspective of ABE providers, their programs are a constant in the community, and they 
offer valuable academic support services for companies. Companies need to know that ABE 
programs exist and what they can offer. (ABE) 
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V. OVERVIEW OF FINDINGS 



' , ■ ■ • Program Development ■ • ; 

Participant and instructor evaluations of STAT courses and workshops indicated that 
participants gave high ratings to the training activities and value the opportunities provided by 
the program. This was reinforced by classroom observations, participant responses to the 
Interim Project Evaluation Questionnaire, and comments made by focus group participants. 

• Classes observed were extremely Well planned and well received. Objectives were 
clear and there was a high level of interest, participation, and attention to task on 
the part of participants. All instructors were well prepared, warm, friendly, and 
accepting. 

• The mean response to item 5 of the Interim Project Evaluation Questionnaire was 
4.21, indicating that instructors, instructional methods, and materials were 
perceived by participants to be of high quality. 

• Focus group participants found the classes helpful in developing new skills and 
preparing for the Selection Exercises. Instructors were highly rated, and the fact 
that they were all external to BCBSMo was seen as a strength. 

Other perceived strengths of the training program include the following: flexible 
scheduling; small class size; voluntary attendance; availability of staff; various teaching styles; 
support from some managers for attending classes; support from classmates with similar 
needs; and individual assistance. While participants felt that the training was probably helpful 
in improving their on-the-job skills, the relationship was less obvious. 

When asked to identify ways in which the program could be improved, individual 
participants and supervisors gave the following suggestions: 

• Start some college studies. 

• Offer more advanced material once you meet company standards in Selection 
Exercises, Phase I. 

• Provide more Saturday or after-work sessions. 

• Have more sessions. 

• Make sure materials are error free. 
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• Hold training during working hours. 

• Provide more test-taking practice. 

• Make sure program mailings get to workers. 

• Have a class on office etiquette for everyone. 

• Include supervisors/managers in the information loop 

• Provide supervisors/managers with feedback on program progress 

• Provide managers with an overview of skills taught 

• Include an overview of STAT as part of the orientation for new employees 

• Market the program as a vehicle for personal/career development rather than 
remediation 

• Change the project name to remove the association with remediation 

• Separate professional development from selection exercises 

• Provide incentives for participation 

• Advertise program and individual accomplishments— accomplishment board, 
testimonials from participants, newsletter 

• Provide a second level of skills upgrades for employees above the clerical level 

• Reach production workers 

• Videotape classes for home study use 

• Continue the use of practice tests 

• Provide classes in sales, communication, time management, organizational skills 

• Have brown bag lunches 

Three external reviewers and two internal reviewers reviewed the following curriculum 
packets: Perception/Concentration; Comprehension/Problem Solving; Memory Skills; Math; 
and Writing. Each packet was reviewed for relevancy of the topic, quality of organization, 
appropriateness of reading level, and transferability. In general, the reviewers gave high 
ratings to the materials. 

BCBSMo administrators interviewed reported that the federal grant and three-member 
partnership has benefited the company by increasing the scope of the program and focusing 
additional resources and expertise on the effort. They also felt that the program had a 
tremendous confidence-building effect on participants, with some are even planning to 
continue their educations. This has in turn helped morale and productivity and is in the best 
interest of the company. Other strengths of the program, according to these administrators, 
include instructors being from outside the company, the sensitivity of instructors, and the 
voluntary nature of the program. 
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BCBSMo administrators expressed some disappointment that the external component 
has not worked as planned, however, they explained that the employment needs of the 
company changed over the last few years, and an external pool of qualified applicants is less 
critical than it was previously. 



ho|>j< I «»!! h>.h* jjjfgjj \\*>r\ || 

The program benefited individual workers in several ways, including increased skills, 
increased scores oh the Selection Exercises, and increased self-confidence. 

• Participating workers showed significant increases in basic skills acquisition after 
completing STAT courses in reading, math, and punctuation/grammar. Based on 
pre-post gain scores: 95 percent improved their reading skills; 86 percent showed 
improvements in math skills; 81 percent improved their punctuation /grammar 
skills; and 85 percent showed overall increases in basic skills acquisition. 

• From 64 percent to 88 percent of STAT participants improved their scores on the 
Selection Exercises-88 percent improved their scores on the perception/ 
concentration section of the exercises, 80 percent improved their scores on the 
memory subtest, 70 percent improved their scores in numerical computation and 
written expression, and 64 percent improved in the area of comprehension/problem 
solving. 

• Sixty-three percent of STAT participants who re-took one or more sections of the 
Selection Exercises subsequently met company standards for continued 
employment and eligibility to compete for promotions. Furthermore, for the 
subtests of perception /concentration, written expression, and 
comprehension /problem solving, participants who completed a related STAT 
course before retaking one or more sections of the test received significantly higher 
gain scores than those who did not. 

• Ninety-two percent of participants felt that participation in the program had 
increased their self-confidence; 60 percent indicated that it had increased their self- 
confidence to a great extent. This was reinforced by participant and supervisors 
comments in the focus groups conducted as part of the formative evaluation. 
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Summative evaluation data suggest that some modest improvements in employee 
performance and productivity as well as increased customer satisfaction ratings may be 
attributable in part to program participation. 

• From 1991 to 1994, the percentage of STAT participants whose performance ratings 
improved (49%) was considerably greater than the percentage whose scores 
declined (20%) or stayed the same (31%). 

• A considerably higher proportion of STAT participants received improved 
productivity ratings (46%) over a four-year period as compared to those whose 
ratings declined (4%). Fifty percent of the ratings stayed the same from 1991 to 
1994. 

• Over a four-year period, 42 percent of STAT participants sampled in the customer 
service department received improved customer satisfaction ratings, while none 
received lower ratings. The ratings for 58 percent stayed the same. 

According to BCBSMo administrators: 

• The project has made a difference at Blue Cross and Blue Shield of Missouri. It has filled a 
need to provide academic support to no7i-managerial staff, and had helped to change the 
nature and importance of employee input into company operations—the company wiV 
conduct employee climate surveys, all-employee meetings will be held, and employee 
feedback groups will be instituted. By the end of April 1995, every employee will have a 
development plan. 

• Employees need developmental support, want it, and STAT has played an important role. 
This support needs to be provided into 1995 and beyond. It may become a more internal 
process in the future. 

Finally, the program has been institutionalized. Starting in January of 1995, STAT 
workshops will be offered on company time. The program has been restructured from a 
classroom instruction model to a more individualized, self-paced, computer-assisted approach 
provided in a multi-station basic skills lab. Blue Cross and Blue Shield of Missouri purchased 
all of the hardware for the lab and some of the software, and the grant supported the costs of 
additional basic skills software. The system is now being accessed by other Blue Cross and 
Blue Shield offices in Missouri. 
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Course Catalog 

Fall 1993 
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WHAT IS STAT? 

Skills Today for Advancement Tomorrow 



STAT is an educational partnership of BlueCross BlueShield of Missouri. 
St. Louis Community College, and St. Louis Public Schools. 



STAT OFFERS: 

FREE educational services for BCBSMo employees 
f/ Get ready for college courses 
1/ Brush up on Writing, Math. Memory Skills, etc. 
i/ Prepare for taking or re-taking Selection Exercises 

Individual tutoring 

Take-home study packets 

Educational counseling 

The STAT staff, all experienced St. Louis Community College teachers, 
seek to meet your educational needs and varied working schedules. 

We offer classes /workshops at lunchtime, after work, and some Saturday 
mornings. Begin your future and register for STAT. 

REGISTRATION - Mail in your registration by August 12. or. if you have 
registration problems or questions, register in person at the following 
locations: 

BCBSMo Plaza 
August 11th 
Missouri Room 
1 1:00 am - 2:00 pm 

One City Centre 
August 12th 
8th floor HR conference room 
11:00 am - 2:00 pm 



QUESTIONS? Call Loma Finch (ext. 34779). Pam Richards, (ext. 34394). 

or Carolyn Martini (ext. 34050) 
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STAT CLASSES 



Reading Skills Brush-Up — Overview of reading strategies, including reading tables 
& graphs, comprehension, application, and drawing conclusions. All course 
materials will be supplied. 

Math Skills Brush-Up — Practice on math concepts and computation exerciser in 
whole numbers, signed numbers, fractions, decimals, percents. and word problems. 
All course materials will be supplied. 

Punctuation &. Grammar Skills — Brush up on business English, including spelling, 
punctuation, capitalization, grammar, and word usage. Sharpen your editing and 
proofreading skills. All course materials will be supplied. 

Business Writing Skills — Introducing the POWER writing method: 

Elanning Organizing Writing Evaluating Revising 

Learn to write power letters, memos and reports. Punctuation and Grammar Skills 
class strongly recommended to prepare for this class. All course materials will be 
supplied. 



STAT WORKSHOPS 

Effective Communication Skills — Four-session workshop covering essential 
communication skills — questioning, listening, responding, and presenting. Learn to 
say what you mean! (4 hours) 

Word Power — Looking for just the right word? This workshop will help you learn to 
increase your vocabulary in just two sessions. (2 hours) 

Improve Your Memory — Two sessions on memory tecrniques. Useful tips on 
memory improvement and practice exercises will be covered. (2 hours) 

Improve Your Concentration — It's all a mind game and you can master some game 
rules in just one session. (1 hour) 

Problem Solving in two sessions leam to approach a problem in a logical, 
systematic way. Practice the six basic steps to break a problem into manageable 
parts. (2 hours) 

Study Skills and Test Anxiety— Leam a system in just two sessions that not only will 
help you make better use of your study time but also will cut down on your test 
anxiety. (2 hours) 

Introduction to the Personal Computer — Basic information on the PC including 
keyboard layout, etc. You will also receive a brief introduction to some common PC 
applications. (DATES TO BE ANNOUNCED) 



BEST COPY AVAILABLE 
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STAT SELF-STUDY PACKETS 



Business Letter-Writing Exercises — Self-study practice on writing efficient, effective 
business letters. (The practice letters may be reviewed by an instructor at the 
employee's request. ) 

Perception/Concentration Skills — Self-study exercises to provide practice in 
proofreading, code matching, and attending to details. 

Memory Skills — Self-study exercises to provide practice in workplace memory and 
application skills. 



One-on-one assistance will be available for all STA1 
classes, workshops, and self-study packets. 
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STAT REGISTRATION FORM 



Name: 


Department: 




Mail Route: 


Extension: 








Please sign me up for the STAT class(es) checked below: 



Reading Skills Brush-Up : 

Section Place Davs 

□ R01 1831 Chestnut Tue/Thu 

□ R02 1831 Chestnut Mon/Wed 

□ R03 1831 Chestnut Mon/Wed 

□ R04 One City Centre Tue/Thu 



Time 

4:30-6:00 pm 
11:00- 12:00 pm 

12:00- 1:00 pm 
11:30- 12:30 pm 



Dates 
8/3 1 . 9/23/93 
8/30- 10/8/93 
8/30- 10/8/93 
8/31 - 10/7/93 



Math Skills BrushUp : 

Section Place 

□ M01 1831 Chestnut 

□ M02 1831 Chestnut 

□ M03 1831 Chestnut 

□ M05 One City Centre 

Punctuation Si Grammar Skills : 

Section Place 
Q poi 1831 Chestnut 

□ P02 1831 Chestnut 
Q po3 1831 Chestnut 
Q po4 One City Centre 



Day's 
Mon/Wed 

Tue/Thu 

Tue/Thu 

Mon/Wed 



Davs 
Mon/Wed 
Tue/Thu 
Tue/Thu 
Mon/Wed 



Time 

4:30 - 6:00 pm 
11:00- 12:00 pm 
12:00- 1:00 pm 
12:30- 1:30 pm 



Time 

4:30 - 6:00 pm 
11:00- 12:00 pm 
12:00- 1:00 pm 
11:30- 12:30 pm 



Dates 
8/30.9/27/93 

8/31 - 10/7/93 
8/31 - 10/7/93 
8/30- 10/8/93 



Dates 
8/30 . 9/27/93 
8/31 - 10/7/93 
8/31 - 10/7/93 
8/30 - 10/8/93 



Business Writing Skills : Please note Business Writing start dates. Punctuation & 
Grammar class is strongly recommended before enrolling in Business Writing. 



Section 

□ W01 

□ W02 

□ W03 

□ W05 



Place 
1831 Chestnut 
1831 Chestnut 
1831 Chestnut 
One City Centre 



Days 
Mon/Wed 
Tue/Thu 
Tue/Thu 
Mon/Wed 



Time 
4:30 - 6:00 pm 
11:00- 12:00 pm 
12:00- 1:00 pm 
*2:30- 1:30 pm 



Dales 

10/11 - 11/3/93 
10/12 - 11/18/93 
10/12 - 11/18/93 
10/11 - 11/17/93 



Please sign me up for the STAT workshop(s) checked below 



Effective Communication: 

Section Place 

□ 01 1831 Chestnut 
Q 02 1831 Chestnut 

□ 03 One City Centre 
Q 04 1831 Chestnut 
Q 05 One City Centre 



(4 Hours) 

Mon, Tue, Wed, Thur 
Sat. 

Mon, Tue, Wed, Thur 
Mon, Tue, Wed. Thur 
Mon. Tue. Wed. Thur 



Time 
12:00- 1:00 pm 
8:30 am- 12:30 pm 
12:00- 1:00 pm 
11:00- 12:00 pm 
11:30- 12:30 pm 



Dates 
9/13-9/16/93 

9/18/93 
9/20 • 9/23/93 
10/11 - 10/14/93 
10/18 • 10/21/93 



Return Registration Form to Sally Bartsokas, HRD - 0004, By August vi 
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Word Power: 

Section 

□ 01 

□ 02 



(2 Hours) 

Place 
1831 Chestnut 
One City Centre 



Improve Your Memory : (2 Hours) 
Section Plage, 

□ 01 1831 Chestnut 

□ 02 One City Centre 

□ 03 1831 Chestnut 

□ 04 One City Centre 



Pay? 

Mon. Tue 
Wed. Thur 



Day? 
Mon. Tue 

Mon. Tue 

Mon. Tue 

Wed. Thur 



Improve Your Concentration: (2 Hours) 



Section 

□ 01 

□ 02 

□ 03 

□ 04 

Problem Solving: 

Section 

□ 01 

□ 02 
□' 03 

□ 04 



Place 
1831 Chestnut 
One City Centre 
1831 Chestnut 
One City Centre 

(2 Hours) 
Place 
1831 Chestnut 
One City Centre 
1831 Chestnut 
One City Centre 



Study Skills and Test Anxiety: 
Section Place 

□ 01 1831 Chestnut 

□ 02 One City Centre 

□ 03 1831 Chestnut 



□ 04 



One City Centre 



Pays 

Mon. Tue 
Wed. Thur 
Mon. Tue 
Wed. Thur 



Pay? 

Wed. Thur 
Mon. Tue 
Mon. Tue 

Wed. Thur 

(2 Hours) 

Das 

Tue 
Wed 
Tue 
Wed 



Time 
11:00- 12:00 pm 
11:30- 12:30 pm 



Time 
12:00- 1 
12:00- 1 
11:00- 12 
11:30- 12 



00 pm 

:0O pm 
;00 pm 
30 pm 



Time 

11:00- 12 
11:30- 12 
12:00-1 
12:30- 1 



00 pm 
30 pm 
:00 pm 
30 pm 



Time 

11:00- 12:00 pm 
11:00- 12:00 pm 
12:00 - 1:00 pm 
11:30- 12:30 pm 



Time 

12:00 - 1:00 pm 
12:00- 1 :00 pm 
11:00- 12:00 pm 
11:30 - 12:30 pm 



Dates 
10/11 - 10/12/93 
10/13- 10/14/9? 



Dates 
9/13-9/14/93 

9/20-9/21/93 

10/4- 10/5/93 

10/6- 1 0^7/93 



Dates 

9/27 - 9/28/93 
9/29 - 9/30/93 
10/11 - 10/12/93 
10/13- 10/14/93 



Dates 
9/22 - 9/23/93 
9/20 - 9/21/93 
10/4 - 10/5/93 
10/6 - 10/7/93 



Dates 

9/14/93 

9/15/93 

10/12/93 

10/13/93 



Please send me the STAT self-study packet(s) checked 
below: 

□ Business Letter-Writing EZxercises Packet 



□ Perception/Concentration Skills Packet 



□ Memory Skills Pocket 
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Guidelines for Formative Evaluation 
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APPENDIX C. 
Classroom Observation Form 
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STATPROjECT 
TRAINING OBSERVATION FORM 



Course/Workshop:. 



Date /Times/Location: 



.No. of Participants: 



Clarity of objectives: 

Level of interest of participants: 

Level of participation of participants: 

Level of attention to trs^ of participants: 

Quality/appropriateness of teaching methods: 

Quality of instructional materials: 



High 

5 

5 

5 

5 

5 

5 



4 
4 
4 
4 
4 
4 



3 
3 
3 
3 
3 
3 



Low 



2 
2 
2 
2 
2 
2 



Description of Activities: 
Time Period #1 (10 minutes) 



Time Period #2 (10 minutes) 



Time Period #3 (10 minutes) 
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APPENDIX D. 
Interview Format 
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INTERVIIEW QUESTIONS 
STAT PROJECT INTERIM EVALUATION 



(1) To what extent are the project goals and objectives clear? 

(2) To what extent are the project's activities related to stated goals and objectives? 

(3) To what extent is the project well organized? 

(4) To what extent are lines of communication open, and do members of the 
partnership feel comfortable sharing information with one another? 

(5) To what extent are recruitment strategies reaching the BCBSMo workers for 
whom they are intended? 

(6) To what extent are workers responding to the project in sufficient numbers? 

(7) To what extent are training activities successful in improving workplace 
literacy skills? 

(8) How might training activities be improved? 

(9) What additional support services are available to participants? 

(10) To what extent do the participants find the services helpful? 

(11) To what extent do BCBSMo workers participate in project planning and review? 

(12) To what extent are the project's dissemination activities effective in increasing 
awareness of the need for and value of workplace literacy? 

(13) What are the project's greatest strengths? 

(14) How might the project be improved? 
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APPENDIX E. 
Interim Project Evaluation Questionnaire 
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MEMORANDUM 

DATE: February 16, 1994 

TO: Current and past STAT participants 

FROM: STAT staff 

RE: STAT interim evaluation 



We want to evaluate Human Resources Development's STAT (Skills 
Today /or Advancement Tomorrow) program, and we need your 
feedback. 

Please take a few moments to complete the enclosed survey. We 
appreciate your comments as this helps us design classes and 
learning. materials to meet your needs and interests. We will share 
the results of this evaluation with you. 

Return the survey to Pam Richards, HRD-8004, by February 25. 

We look forward to hearing from you. Thank you. 
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calls Today for Advancemen t 



Torfiorr fyw 



INTERIM EVALUATION FORM 

Please take a few minutes to complete the following brief questionnaire. Your ratings and 
comments will help us strengthen STATs strategies and activities. 



(1) To what extent are STATs goals and objectives clear? 

(2) To what extent is STAT well organized? 

(3) To what extent are recruitment strategies effective? 

(4) To what extent do training activities match real needs? 

(5) To what extent are instructors. Instructional 
methods, and materials of high quality? 

(6) To what extent are training activities scheduled 
at convenient times? 

(7) To what extent are training activities successful 
in improving workplace literacy skills? 

(8) To what extent are STAT support services, such as Skills 
Center, self-study exercises, etc.. helpful to participants? 

(9) To what extent have you taken advantage of these services? 

(10) To what extent have you found these services helpful? 

(11) To what extent has your participation in STAT increased 
your self confidence? 

(12) To what extent has your participation in STAT helped you 
to become a better worker? 

(13) In what ways can STAT be improved? 



To a 

Great Extent 



5 
5 
5 
5 
5 

5 

5 

5 

5 
5 
5 



4 
4 
4 
4 
4 



4 
4 
4 



3 
3 
3 
3 
3 

3 

3 



3 
3 
3 



Not a 
All 



2 
2 
2 
2 
2 

2 

2 

2 

2 
2 
2 



Other Comments: 



Return to Pam Richards, HRD-8004, by February 25, 1994 
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O R A N D U M 



DATE: 



February 16. 1994 



TO: 



Unit managers and managers 



FROM: 



STAT staff 



RE: 



STAT int :rim evaluation 



We want to evaluate Human Resources Development's STAT (Skills 
Today for Advancement Tomorrow) program, and we need your 
feedback as a supervisor. 

Please take a few moments to complete the enclosed survey. Your 
comments will help us measure how well we reach and meet the 
needs of BCBSMo's employees. We will share the results of this 
evaluation with you. 

Return the survey to Pam Richards, HRD-8004, by February 25. 

We look forward to hearing from you. Thank you. 
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INTERIM EVALUATION FORM 



Please take a few minutes to complete the following brief questionnaire. Your ratings and 
comments will help us strengthen STATs strategies and activities. 

To a Not at 
Great Extent All 

(1) To what extent are you aware of the STAT project? 5 4 3 2 

(2) To what extent are STATs goals and objectives clear? 5 4 3 2 

(3) To what extent is STAT well organized? 5 4 3 2 

(4) To what extent do STAT staff seem to be well qualified? 5 4 3 2 

(5) To what extent are recruitment strategies reaching the 5 4 3 2 
BCBSMo workers for whom they are intended? 

(6) To what extent are training activities successful in 5 4 3 2 
improving workplace literacy skills? 

(7) To what extent are the other STAT support services .such as 5 4 3 2 
Skills Center, self-study exercises, etc.. by STAT helpful 
to participants? 

(8) How might STATs activities b»; improved? 



Other Comments? 



Return to Pam Richards, HRD-8004, by February 25, 1994 
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APPENDIX F. 

Focus Group Guidelines and Discussion Form for Participants 
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GUIDELINES FOR FOCUS GROUP LEADERS 



(1) Have participants take a few minutes to complete the Focus Group Discussion Form. Be sure 
to tell them that it is an exercise designed to help them organize their thoughts about the 
program and that the completed forms will not be collected. 

(2) Have participants spend approximately 15 minutes discussing ways in which they have 
benefitted from the program. Have a recorder list their comments on newsprint or a chalk 
board. 

(3) Use the same process to identify and discuss perceived strenghts (15 minutes) and weaknesses 
(15 minutes) of the program. 

(4) Make a quick assessment of the extent to which responses have addressed intended 
questions below (all questions will not be appropriate for all groups). Spend the last 15 
minutes directing a few specific questions to the group designed to solicit discussion of the 
points not covered earlier. 

• Project Goals and Objectives 

Are the project goals and objectives clear? 

To what extent do all partners have a common understanding of goals and objectives? 
To what extent are the project's activities related to stated goals and objectives? 
Are the objectives measurable? 

• Project Organization . 

Is the project well organized? 

Are roles and responsibilities clear and understood by all members of the partnership? 
Are lines of communication open, and do members of the partnership feel comfortable 

sharing information with one another? 
Are project staff well trained? 
Is the project operating on schedule? 

• Recruitment and Selection of Participants 

Are recruitment strategies reaching the BCBSMo workers for whom they are intended? 

Are workers responding to the project in sufficient numbers? 

Are selection criteria well defined and appropriate? 

How might recruitment and selection procedures be improved? 

• Participant Training 

Do training activities match real needs? 

Are instructors, instructional methods, and materials appropriate? 
Are training activities scheduled at convenient times? 
Are training activities successful in improving workplace literacy skills? 
How might training activities be improved? 
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Participant Support Services 

What other support services are available to participants? 

To what extent have participants taken advantage of these services? 

Do the participants find the services helpful? 

What other support services do participants need? 

Peer Involvement in Project Planning and Review 

In what ways have other BCBSMo workers participated in project planning and review? 
What is the nature of their participation? 
How might this component be strengthened ? 

Dissemination 

To what extent is information on project activities and outcomes disseminated to the 
community? 

Are dissemination activities effective in increasing awareness of the need for and value 

of workplace literacy? 
How can dissemination strategies be improved? 

Overall Benefits for Participants 

In what ways have participants benefitted from the program? 
Personally? In their jobs? 

Overall Strengths and Weaknesses 

What are the overall strengths of the program? 
How can the program be improved? 
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rrr? 









(1) In what ways have you benefitted from the STAT program? 



(2) What are some of the greatest strengths of the program? 



(3) In what ways could the program be improved? 
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APPENDIX G. 
Focus Group Discussion Form for Supervisors 
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(1) How familiar are you with the STAT Program? 

(2) Have any of your employees participated in STAT? 

(3) In what ways do you think employees have benefitted from the STAT program? 



(4) To what extent has STAT met your needs as a supervisor? 

(5) What are some of the greatest strengths of the program? 



(6) In what ways could the program be improved? 



103 



APPENDIX H. 
Guidelines for Summative Evaluation 
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Additional Comments: Interim Project Evaluation Questionnaire 
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T "** 1 1 . 1 M ' ™ 1 « . , ' . '■ ■ I — 1 . ! ■ II. ■ , ■■ n ,| , , 

>-^-l^lj^j>'jjL-'^^^ nts; tiiterlm Project Evaluation Questionnaire . : 
Comments and Suggestions: STAT Participants 

In what ways can STAT be improved? 

• Don't know. 

• I think it's Great. Start some college studies. I need it. 

• The only suggestion I had was not have back-to-back classes such and Math (lunch time) & 
English during same week period. To my knowledge it has already been changed. Thanks 
again. 

• More Saturday or after-work sessions. 

• Maybe more advanced material once you pass the first phase. 

• Have more sessions-don' I try to rush or cram more into short sessions, sometimes 2 or 3 
days are not enough. Also, would prefer more lunch hour sessions. 

• Would prefer having STAT classes at One City Center. 

• Allowing math students to use calculators during class/especially testing. 

• One way is that you can improve by letting the test Start as you finish one part of the test. 
And let those people who may have gotten a low acceptable in one part take that part ova- 
right away and slop making everyone way a whole year before they can take it again. 

• Self study exercises are very helpful but frustrating when you don't agree with the answers 
and they also don't tell you why their answer is correct. 

• Get more classes going! 

• Classes during my lunch hour might enable me to be able to attend. Still have backlog in my 
department. 

Other Comments 

• Training activities should not be on our lunch hour or after work. It should be during 
■working hours. I feel we would benefit better during working hours not after when your 
tired from a longs day work or at lunch when your hungry. You cannot eat and do the 
training exercises at the same time. 

• / have not yet utilized your services. Will be though. 

• Management should be more understanding of people needing lunch lime for classes. In 
Client Service, they do not want to help you attend classes!! 

• Since my participation in STAT so far has been minimal-I hope to get a chance to reevaluate 
after I attend workshops I scheduled in March! 
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The memory program should be related to the actual STAT exercise. 

The Instructor-frating of 5); the memory class was fun and relaxing-(raHng of 5); and the 
presentation was great-(rating of 5). 

Extend the time for the workshops, materials that needs to be covered is too rushed. 

I need to take/find the time to use the program to benefit from these services. 

I only briefly had contact with STAT personnel, and they helped me in a more guidance type 
of situation. I haven't participated in any of the training activities. 

The only thing I've noticed is the classes seem to dwindle in size, but I don't know why a lot 
of people lose interest. I did not. A class of about 12 people dwindled to 2 by the end of the 
class. 

You are very helpful, but it's still hard for me to understand the problem, maybe I am to 
hard on myself. 

I feel that the negative connotations associated with the STAT program are detrimental to its 
success. If an employee feels they are going to lose their jobs (and that is what is being 
expressed by employers) based on the testing and skills assessment, it is highly unlikely that 
their capacity to learn is at its best, stress is a detriment to learning. Unfortunately, I don 't 
think the STAT program has control over the employee impressions and that comes from 
H.R. J 

The instructor really made the difference. He is the best instructor I have had teach a course 
thus far at BCBS. 

Would like to have more help deciding on greater education. 

If managers realized employees were taking the extra effort to participate in STAT, they 
would realize their desire to grow and learn. I know STAT has had a positive effect on my 
job. 

That Memory Test should be improved by giving a person more time to study the keys for 
the test. That part is ridiculous for studying 10 mins. Also need to improve on the letter 
skills for as you know what to right down but its not the way that the instructors want you 
to go about it. 

Improve and/or add additional. 

I appreciate the STAT classes and help. I just wish I had more time to take some courses. 
I've been through them, I'm waiting for some new ones. 

I would like to see classes available at One City Center more, especially the evening classes. 

Keep offering the courses for employees who are sincere about imp, oving their work skills 
and education. 

The literature is hard to read in some of the books and answers are wrong, ivhich can be 
discouraging to someone who thought they where right and saw the wrong answer. 
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Good program-helpful to employee. Currently involved in completing other courses but 
plan to take advantage of program when finished. 

You cannot expect a person to learn from alwk course what it takes a semester to teach. 
The pressure is too great and the time between the classes and test is even greater. Everyone 
is not test material but has the knowledge to do their job very well. Why should a person be 
expected to take a test to keep a job that they know instead of getting another one? 



Training at lunch time is fine, but, I can't always make each class, because of special projects 
that are assigned to me. Maybe if there could be some training classes Saturday mornings 
(That is just a suggestion), or during the working hours. 



I was always afraid to sign up for any classes fear of appearing stupid or dumb but after 
taking the memory class I am ready to sign up for more classes. I felt very relaxed and knew 
I was not the only one with mixed feelings. Pam was an excellent instructor and made the 
class fun. I learned a lot from her. 



More "Core Classes" during lunch times. If possible add breaks. Possibly more Saturday 
mid-morning classes (core classes). 



I think that the STAT program is great! 



Comments and Suggestions: Supervisors 



How might STATs act ivities be improved? 

• I have a new unit since mid-December. I would like to know what level was established for 
certain job classificatiotis. I guess what I am asking for is that the criteria for all jobs be 
published. 

• Need to know more about the program to comment. 

• As there is minimal communication from the employee to the supervisor on this I cannot 
answer these questions. And I myself have not taken any STAT classes. 

• I think that the STAT program has a very negative reputation, not due to their own actions 
but rather because Human Resources has given erroneous information regarding the tests 
that are given (Phase I & II). I'm sure if the STAT program were offered without the 
threatening undercurrents of job loss etc., it would be received much more readily and in the 
long run accomplish more for the company and the individuals thai need skills enhancement. 
Human beings respond much better to positive reinforcement'. 

Other Comments 

• Not sure who STAT staff are. 

• Aren 't reaching the people needed to reach. 

• Know * individuals in our department who have found it very useful. 

• I wasn't aware of program till October 93. 

• Other questions unable to answer due to no experience working with the program. 
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APPENDIX J. 
Curriculum Analysis Forms 
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ANALYSIS FORM: PERCEPTION/CONCENTRATION SKILLS CURRICl II I m 



Rate each of the perception/concentration skills curriculum components on the following criteria- 
relevancy of topic; quality of organization; appropriateness of reading level; and transferability. ' 

(1) Accessing Information by Computer Code 



• , High Tow 

Relevancy of Topic: 5 4 3 2 1 

Quality of Organization: 5 4 3 2 1 

Appropriateness of Reading Level: 5 4 3 2 1 

Transferability: 5 4 3 2 1 



Comments: 



(2) Comparing Numbers and Letters 



Relevancy of Topic: 
Quality of Organization: 
Appropriateness of Reading Level: 
Transferability: 

Comments: 



(3) Reading Tables 



High Low 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 



Relevancy of Topic: 
Quality of Organization: 
Appropriateness of Reading Level: 
Transferability: 

Comments: 



High Low 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 
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Rate each of the reading curriculum components on the following criteria: relevancy of topic; quality 
of organization; appropriateness of reading level; and transferability. Assessment instruments are 
rated on the following criteria: appropriateness of items; appropriateness of reading level; and 
transferability. 

(1) Understanding Written Material 

High Low 
Relevancy of Topic: 5 4 3 2 1 

Quality of Organization: 5 4 3 2 1 

Appropriateness of Reading Level: 5 4 3 2 1 

Transferability: 5 4 3 2 1 

Comments: 

(2) Applying Policy 

High Low 

Relevancy of Topic: 5 4 3 2 1 

Quality of Organization: 5 4 3 2 1 

Appropriateness of Reading Level: 5 4 3 2 1 

Transferability: 5 4 3 2 1 

Comments: 



(3) Classifying Subscriber and Provider Inquiries 

High Low 
Relevancy of Topic: 5 4 3 2 1 
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Quality of Organization: 5 4 3 2 

Appropriateness of Reading Level: 5 4 3 2 

Transferability: 5 4 3 2 

Comments: 



(4) Drawing Conclusions 



High Low 

Relevancy of Topic: 5 4 3 2 1 

Quality of Organization: 5 4 3 2 1 

Appropriateness of Reading Level: 5 4 3 2 1 



Transferability: 5 4 3 2 

Comments: 
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(5) Reading Pre Test 

Appropriateness of Items: 
Appropriateness of Reading Level: 
Transferability: 

Comments: 



(6) Reading Post Test 

Appropriateness of Items: 
Appropriateness of Reading Level: 
Transferability: 

Comments: 



9 

ERIC 



125 



Rate each of the memory skills curriculum components on the following criteria: relevancy of topic; 
quality of organization; appropriateness of reading level; and transferability. 

(1) Memory Skills Packet 

High Low- 
Relevancy of Topic: 5 4 3 2 1 
Quality of Organization: 5 4 3 2 1 
Appropriateness of Reading Level: 5 4 3 2 1 
Transferability: 5 4 3 2 1 

Comments: 



(2) Memory Skills Exercises 

High Low 

Relevancy of Topic: 5 4 3 2 1 

Quality of Organization: 5 4 3 2 1 

Appropriateness of Reading Level: 5 4 3 2 1 

Transferability: 5 4 3 2 1 

Comments: 
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: : ; STAT PROJECT 
ANALYSIS FORM: MATH CURRICULUM 



Rate each of the math curriculum components on the following criteria: relevancy of topic; quality of 
organization; appropriateness of reading level; and transferability. Assessment instruments are rated 
on the following criteria: appropriateness of items; appropriateness of reading level; and 
transferability. 



(1) Packet 1, Parts I and II (Whole Numbers) 

High Low 

Relevancy of Topic: 5 4 3 2 1 

Quality of Organization: 5 4 3 2 1 

Appropriateness of Reading Level: 5 4 3 2 1 

Transferability: 5 4 3 2 1 

Comments: 



(2) Packet 2 (Signed Numbers) 

High Low 

Relevancy of Topic: 5 4 3 2 1 

Quality of Organization: 5 4 3 2 1 

Appropriateness of Reading Level: 5 4 3 2 1 

Transferability: 5 4 3 2 1 

Comments: 



(3) Packet 3, Parts I and II (Fractions) 

High Low 
Relevancy of Topic: 5 4 3 2 1 



Quality of Organization: 5 4 3 2 

Appropriateness of Reading Level: 5 4 , 3 2 

Transferability: 5 4 3 2 

Comments: 



(4) Packet 4, Parts I and II (Decimals) 



High Low 
Relevancy of Topic: 5 4 3 2 1 



Quality of Organization: 5 4 3 2 

Appropriateness of Reading Level: 5 4 3 2 

Transferability: 5 4 3 2 

Comments: 
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Rate each of the writing curriculum components on the following criteria: relevancy of topic; quality 
of organization; appropriateness of reading level; and transferability. Assessment instruments are 
rated on the following criteria: appropriateness of items; appropriateness of reading level; and 
transferability. 

(1) Packet 1 (Spelling, Punctuation, Capitalization) 

High Low 
Relevancy of Topic: 5 4 3 2 1 

Quality of Organization: 5 4 3 2 1 

Appropriateness of Reading Level: 5 4 3 2 1 

Transferability: 5 4 3 2 1 

Comments: 



(2) Packet 2 (Punctuation) 

High Low 

Relevancy of Topic: 5 4 3 2 1 

Quality of Organization: 5 4 3 2 1 

Appropriateness of Reading Level: 5 4 3 2 1 

Transferability: 5 4 3 2 1 

Comments: 



(3) Packet 3 (Grammar) 

High Low 

Relevancy of Topic: 5 4 3 2 1 

Quality of Organization: 5 4 3 2 1 

Appropriateness of Reading Level: 5 4 3 2 1 

Transferability: 5 4 3 2 1 

Comments: 



(4) Packet 4 (Grammar) 

High Low 

Relevancy of Topic: 5 4 3 2 1 

Quality of Organization: 5 4 3 2 1 

Appropriateness of Reading Level: 5 4 3 2 1 

Transferability: 5 4 3 2 1 

Comments: 
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(5) Packet 5 (Word Usage) 

High Low 

Relevancy of Topic: 5 4 3 2 1 

Quality of Organization: 5 4 3 2 1 

Appropriateness of Reading Level: 5 4 3 2 1 

Transferability: 5 4 3 2 1 

Comments: 

(6) Packet 6 (Cumulative Review) 

High Low 

Relevancy of Topic: 5 4 3 2 1 

Quality of Organization: 5 4 3 2 1 

Appropriateness of Reading Level: 5 4 3 2 1 

Transferability: 5 4 3 2 1 

Comments: 



(7) Writing Pre Test 



High Low 

Appropriateness of Items: 5 4 3 2 1 

Appropriateness of Reading Level: 5 4 3 2 1 

Transferability: 5 4 3 2 1 



Comments: 



(8) Writing Post Test 



Appropriateness of Items: 
Appropriateness of Reading Level: 
Transferability: 



High 








Low 


5 


4 


3 


2 


1 


5 


4 


3 


2 


1 


5 


4 


3 


2 


1 



Comments: 
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APPENDIX K. 
Summary: STAT Partners' Meeting 
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Skills Today for Advancement Tomorrow (STAT) Program 
Partner's Meeting Agenda 
December 6, 1994; 9:00 - 11:00 a.m. 
Alliance Blue Cross and Blue Shield 
8th floor Human Resources 
Chestnut Street Building 



Review Project Outcomes 

Review evaluator's brief abstract of outcomes; 
Coordinators' Reports 

Shared Perceptions of the Program by Each Partner 

(what worked, what didn't work, what would you do again, 
what would you do differently) 

Future Partnership Activities: 
Collaboration 

(Partners' perceptions of future needs, especially 
involving one another in meeting training needs at Alliance 
BCBSMO) 
Dissemination 

(How to contribute /share results and gleanfrom other 
resources within the community) 

Other Business 
Adjourn 
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SUMMARY 
STAT PARTNERS' MEETING 
December 6, 1994 



A meeting of STAT partners was held on December 6, 1994, from 9:00 a.m. to 
11:00 a.m., at Blue Cross and Blue Shield of Missouri. Attending were: 

Wilma Sheffer, Project Director, St. Louis Community College 

Randolph Scott, Vice President for Human Resources, Blue Cross and Blue 

Shield of Missouri 
Del Hemsley, Director of ABE, St. Louis Public Schools 
Lorna Finch, Project Coordinator, St. Louis Community College 
Carolyn Martini, Project Coordinator, Blue Cross and Blue Shield of Missouri 
Jeanne Bumberry, Project Teacher and Resource Assistant, St. Louis Community 

College 

Sharon Slane, External Evaluator 

A copy of the meeting agenda is attached. Four major topics covered were: 
review of project outcomes; shared perceptions of the project by each partner; future 
partnership activities and collaboration; and dissemination. 

Review of Project Outcomes 

The external evaluator gave a brief overview of the evaluation findings to date, 
including the following: 

• Classes observed were extremely well planned and well received. Objectives 
were clear and there was a high level of interest, participation, and attention 
to task on the part of participants. All instructors were well prepared, warm, 
friendly, and accepting. 

• The mean response to item 5 of the Interim Project Evaluation Questionnaire 
was 4.21, indicating that instructors, instructional methods, and materials 
were perceived by participants to be of high quality. 

• STAT participants attending focus groups found the classes helpful in 
developing new skills and preparing for the Selection Exercises. Instructors 
were highly rated, and the fact that they were all external to BCBSMo was 
seen as a strength. 

• Supervisors who attended a focus group felt that the project had been of 
great benefit to participating employees and that it should be expanded and 
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be more visible as a professional development strategy. They also felt that 
supervisors should play a much greater role in project planning and ongoing 
implementation. 

Other perceived strengths of the project include: flexible scheduling; small class 
size; voluntary attendance; availability of staff; various teaching styles; support from 
some managers for attending classes; support from classmates with similar needs; and 
individual assistance. While participants felt that the training was probably helpful in 
improving their on-the-job skills, the relationship was less obvious. In this context, data 
on attendance, punctuality, supervisor performance ratings, productivity, and customer 
satisfaction are currently being analyzed in hopes of linking project participation and 
performance outcomes. 

Shared Perceptions of the Project by Each Partner 

Partners discussed their perceptions of what worked, what didn't work, and 
what they would do differently. The following comments and suggestions were 
generated 1 : 

• Tfie project has made a difference at Blue Cross and Blue Shield of Missouri. It has 
filled a need to provide academic support to non-managerial staff, and had helped to 
change the nature and importance of employee input into company operations-the 
company will conduct employee climate surveys, all-employee meetings will be held, 
and employee feedback groups will be instituted. By the end of April 1995, every 
employee will have a development plan. (BCBS) 2 

• Some things have worked marvclously well, some things haven 't worked as well. If 
we had a chance, we'd do it again. Could we have done more? Probably, but we 
didn't. (BCBS) 

• The project was key to helping employees get through a period of reorganization and 
downsizing, and it gave them something they could do about their situations. 
(BCBS) 

• Employees need developmental support, want it, and STAT has played an important 
role. This support needs to be provided into 1995 and beyond. It may become a 
more internal process in the future. (BCBS) 



These comments and suggestions are exact or nearly exact quotes From meeting participants. 
BCBS-Alliance Blue Cross and Blue Shield of Missouri 
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• A strength of the project from the perspective of the educational providers was that it 
was already in place in the company, and additional job audits weren't required. 
(SLCC) 3 

• Targeting services and focusing on a single department [at a time], rather than 
trying to serve everyone [at the same time], might have been more likely to lead to 
measurable impact. However, the company had a need to provide services company 
wide. (SLCC) 

• Next time we might narrow our expectations. (SLCC) 

• While peer involvement in project planning was built into the proposal, it didn't 
work because the culture wasn't right. The educational provider needs to be flexible 
enough to respond to the business culture. (SLCC) 

• It was hoped that the curriculum could be computerized before the end of the project, 
but that didn't happen. (SLCC) 

• The project would have benefited from much more interface and integration with 
managers and supervisors. (SLCC) 

• The external component, designed to train potential workers, didn't get off the 
ground for a number of reasons, including a reorga7iization and hiring freeze at Blue 
Cross and Blue Shield that wasn't anticipated when the grant was initially 
submitted. Other problems encountered may have been related to Missouri's ABE 
structure. If this component had been implemented, it could have had some positive 
spinoffs, including benefits to the community. (ABE) 4 

• [From the perspective of the ABE Program Director] the project was another 
opportunity to broaden the horizons of some clients, but not directly related to 
employability in the community. It is difficult to related general training materials 
to employability. Clients want something that is tied to getting a job [a concrete 
intervention and a definite connection to actual jobs]. (ABE) 

• Today, workers are in transition, and worker training needs to be a lifelong thing. 
Tliis is finally receiving some attention at the federal and state levels. (BCBS) 

• One of the most important questions for employers to consider with respect to 
employee development is "Does the individual continue to grow in the company and 
demonstrate a continuing willingness and ability to learn?" (SLCC) 

• Approaches to workforce literacy should focus on the ideas of developing workers 
who can continue to learn, rather than on just improving basic skill deficits. 
(SLCC) 



St. Louis Community College 

St. Louis Public Schools Adult Basic Education Program 
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Future Partnership Activities and Collaboration 

Wilma Sheffer explained that three reports will be submitted to the federal 
government and the Regional ERIC office-a fiscal report (SLCC), an outcomes report 
(SLCC), and a final evaluation report (external evaluator). Other comments included 
the following: 

• The project needs to be more visible in the community. (BCBS) 

• We can serve as resources for one another, and we need to work together 
collaboratively in the community, and do our own PR work. (BCBS) 

• We will continue to utilize both of the education partners as providers of on-site 
skills enhancement. (BCBS) 

Dissemination 

In addition to national dissemination efforts, dissemination of project 
information will occur through AAIM, United Way, RCCA, LIFT Missouri, and the 
Literacy Service Center serving the St. Louis area and Missouri. Other possibilities 
include issuing a joint press release, having a joint press conference, and participating 
in community talk shows. Other comments included the following: 

• The community needs to be educated about the broader definition of literacy today- 
its not just reading and writing anymore. Workers need to have skills that will 
allow, them to adapt to a changing workplace and become lifelong learners. (SLCC) 

• There is a paucity of understanding about adult literacy in St. Louis companies It 
tsn t perceived as a high need. It 's not clear what will cause the business 
community to wake up. (BCBS) 

• The role of adult basic education in the zvorkplace is not clearly defined. From the 

P thZZ 1 n E Pr 7 dCr5 ' lhCir pr0SmnS arC a C01M »< the community and 
they offer valuable academic support services for companies. Companies need to 
know that ABE programs exist and what they can offer. (ABE) 
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